Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: shed: Add v13 migration to migrate-state #11601

Merged
merged 20 commits into from
Mar 21, 2024

Conversation

magik6k
Copy link
Contributor

@magik6k magik6k commented Jan 25, 2024

Related Issues

Proposed Changes

Additional Info

Current output

REMINDER: If you are running this, you likely want to ALSO run the continuity testing tool!
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-01-25T15:48:24.150+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Started 7 workers
2024-01-25T15:48:24.151+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Result writer started
2024-01-25T15:48:24.151+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Creating migration jobs
2024-01-25T15:48:34.156+0100    INFO    fil-consensus   filcns/upgrades.go:2418 98152 jobs created, 97145 done, 1007 pending after 10.005548708s (9709/s)
2024-01-25T15:48:44.156+0100    INFO    fil-consensus   filcns/upgrades.go:2418 185317 jobs created, 184310 done, 1007 pending after 20.00564325s (9213/s)
2024-01-25T15:48:54.157+0100    INFO    fil-consensus   filcns/upgrades.go:2418 301311 jobs created, 300304 done, 1007 pending after 30.006194333s (10008/s)
2024-01-25T15:49:04.157+0100    INFO    fil-consensus   filcns/upgrades.go:2418 386255 jobs created, 385248 done, 1007 pending after 40.006319167s (9630/s)
2024-01-25T15:49:14.157+0100    INFO    fil-consensus   filcns/upgrades.go:2418 469984 jobs created, 468977 done, 1007 pending after 50.006376833s (9378/s)
2024-01-25T15:49:24.162+0100    INFO    fil-consensus   filcns/upgrades.go:2418 570946 jobs created, 569939 done, 1007 pending after 1m0.011605083s (9497/s)
2024-01-25T15:49:34.162+0100    INFO    fil-consensus   filcns/upgrades.go:2418 637920 jobs created, 636913 done, 1007 pending after 1m10.011709875s (9097/s)
2024-01-25T15:49:44.162+0100    INFO    fil-consensus   filcns/upgrades.go:2418 728798 jobs created, 727791 done, 1007 pending after 1m20.011806708s (9096/s)
2024-01-25T15:49:54.162+0100    INFO    fil-consensus   filcns/upgrades.go:2418 791869 jobs created, 790862 done, 1007 pending after 1m30.011914208s (8786/s)
2024-01-25T15:50:04.163+0100    INFO    fil-consensus   filcns/upgrades.go:2418 882034 jobs created, 881027 done, 1007 pending after 1m40.012076125s (8809/s)
2024-01-25T15:50:14.163+0100    INFO    fil-consensus   filcns/upgrades.go:2418 956913 jobs created, 955906 done, 1007 pending after 1m50.012562875s (8689/s)
2024-01-25T15:50:24.163+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1033197 jobs created, 1032190 done, 1007 pending after 2m0.012627292s (8601/s)
2024-01-25T15:50:34.163+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1110621 jobs created, 1109614 done, 1007 pending after 2m10.012688625s (8535/s)
2024-01-25T15:50:44.163+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1191577 jobs created, 1190570 done, 1007 pending after 2m20.01278225s (8503/s)
2024-01-25T15:50:54.163+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1255255 jobs created, 1254248 done, 1007 pending after 2m30.01295775s (8361/s)
2024-01-25T15:51:04.164+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1350326 jobs created, 1349319 done, 1007 pending after 2m40.013095875s (8433/s)
2024-01-25T15:51:14.164+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1446653 jobs created, 1445646 done, 1007 pending after 2m50.0132085s (8503/s)
2024-01-25T15:51:24.164+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1536712 jobs created, 1535705 done, 1007 pending after 3m0.013275417s (8531/s)
2024-01-25T15:51:34.164+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1632378 jobs created, 1631371 done, 1007 pending after 3m10.013350958s (8586/s)
2024-01-25T15:51:44.164+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1705779 jobs created, 1704772 done, 1007 pending after 3m20.0134205s (8523/s)
2024-01-25T15:51:54.164+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1797971 jobs created, 1796964 done, 1007 pending after 3m30.013485208s (8556/s)
2024-01-25T15:52:04.164+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1868942 jobs created, 1867935 done, 1007 pending after 3m40.013576375s (8490/s)
2024-01-25T15:52:14.165+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1935723 jobs created, 1934716 done, 1007 pending after 3m50.014086708s (8411/s)
2024-01-25T15:52:24.165+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2006573 jobs created, 2005566 done, 1007 pending after 4m0.014157708s (8356/s)
2024-01-25T15:52:34.165+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2068037 jobs created, 2067030 done, 1007 pending after 4m10.014233208s (8268/s)
2024-01-25T15:52:44.165+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2149619 jobs created, 2148707 done, 912 pending after 4m20.014354417s (8264/s)
2024-01-25T15:52:54.165+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2199829 jobs created, 2198855 done, 974 pending after 4m30.0144465s (8143/s)
2024-01-25T15:53:04.167+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2264154 jobs created, 2263147 done, 1007 pending after 4m40.01603275s (8082/s)
2024-01-25T15:53:14.167+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2314791 jobs created, 2313784 done, 1007 pending after 4m50.0161565s (7978/s)
2024-01-25T15:53:24.168+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2402823 jobs created, 2401816 done, 1007 pending after 5m0.016249333s (8006/s)
2024-01-25T15:53:34.169+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2475169 jobs created, 2474162 done, 1007 pending after 5m10.018313125s (7981/s)
2024-01-25T15:53:44.169+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2565681 jobs created, 2564674 done, 1007 pending after 5m20.018389083s (8014/s)
2024-01-25T15:53:54.169+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2642662 jobs created, 2641655 done, 1007 pending after 5m30.018498542s (8005/s)
2024-01-25T15:54:04.169+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2723608 jobs created, 2722601 done, 1007 pending after 5m40.018577s (8007/s)
2024-01-25T15:54:14.199+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2778931 jobs created, 2777924 done, 1007 pending after 5m50.037615875s (7936/s)
2024-01-25T15:54:24.212+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2849771 jobs created, 2848764 done, 1007 pending after 6m0.037717208s (7912/s)
2024-01-25T15:54:34.219+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2936106 jobs created, 2935181 done, 925 pending after 6m10.037786458s (7932/s)
2024-01-25T15:54:35.165+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Done creating 2947604 migration jobs after 6m10.982977542s
2024-01-25T15:54:35.184+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 1 done
2024-01-25T15:54:35.223+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 4 done
2024-01-25T15:54:35.237+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 6 done
2024-01-25T15:54:35.261+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 0 done
2024-01-25T15:54:35.329+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 5 done
2024-01-25T15:54:36.174+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 2 done
2024-01-25T15:54:36.532+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 3 done
2024-01-25T15:54:36.532+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Result writer wrote 2947603 results to state tree after 6m12.349480875s
2024-01-25T15:54:36.532+0100    INFO    fil-consensus   filcns/upgrades.go:2418 All workers done after 6m12.349460333s
2024-01-25T15:54:36.532+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Running deferred migrations
2024-01-25T15:54:36.532+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Running deferred migration for f05
2024-01-25T15:57:01.796+0100    INFO    fil-consensus   filcns/upgrades.go:2418 All 2947604 done after 8m37.605489417s (5695/s)
migration height  3596225
old cid  bafy2bzaceamqaqlxzhfvs2hf32gvtuqhhfvdbwh7ztrk36foixkimsthdf462
new cid  bafy2bzacedbolm3bvanxqy7b2n5xxovg2eel3ii35qlfunf2iktkteucd5nb2
completed round actual (without cache), took  8m39.557524208s
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-01-25T15:57:03.981+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Started 5 workers
2024-01-25T15:57:03.981+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Creating migration jobs
2024-01-25T15:57:03.982+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Result writer started
2024-01-25T16:02:03.985+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1954661 jobs created, 1954656 done, 5 pending after 5m0.000420667s (6516/s)
2024-01-25T16:04:57.641+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Done creating 2947309 migration jobs after 7m53.655064875s
2024-01-25T16:04:57.642+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 0 done
2024-01-25T16:04:57.642+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 1 done
2024-01-25T16:04:57.743+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 2 done
2024-01-25T16:04:58.312+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 4 done
2024-01-25T16:04:58.516+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 3 done
2024-01-25T16:04:58.516+0100    INFO    fil-consensus   filcns/upgrades.go:2418 All workers done after 7m54.529966834s
2024-01-25T16:04:58.516+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Result writer wrote 2947308 results to state tree after 7m54.52999675s
2024-01-25T16:04:58.516+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Running deferred migrations
2024-01-25T16:04:58.516+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Running deferred migration for f05
2024-01-25T16:07:27.372+0100    INFO    fil-consensus   filcns/upgrades.go:2418 All 2947309 done after 10m23.384064875s (4728/s)
completed premigration, took  10m25.223251917s
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-01-25T16:07:28.955+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Started 7 workers
2024-01-25T16:07:28.955+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Creating migration jobs
2024-01-25T16:07:28.955+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Result writer started
2024-01-25T16:07:38.955+0100    INFO    fil-consensus   filcns/upgrades.go:2418 98152 jobs created, 97145 done, 1007 pending after 10.00006925s (9714/s)
2024-01-25T16:07:48.955+0100    INFO    fil-consensus   filcns/upgrades.go:2418 187297 jobs created, 186290 done, 1007 pending after 20.000144875s (9314/s)
2024-01-25T16:07:58.957+0100    INFO    fil-consensus   filcns/upgrades.go:2418 316593 jobs created, 315586 done, 1007 pending after 30.001249s (10519/s)
2024-01-25T16:08:08.959+0100    INFO    fil-consensus   filcns/upgrades.go:2418 412824 jobs created, 411817 done, 1007 pending after 40.003168166s (10295/s)
2024-01-25T16:08:18.959+0100    INFO    fil-consensus   filcns/upgrades.go:2418 513133 jobs created, 512126 done, 1007 pending after 50.003259916s (10242/s)
2024-01-25T16:08:28.959+0100    INFO    fil-consensus   filcns/upgrades.go:2418 638799 jobs created, 638162 done, 637 pending after 1m0.003432541s (10635/s)
2024-01-25T16:08:38.959+0100    INFO    fil-consensus   filcns/upgrades.go:2418 768513 jobs created, 767506 done, 1007 pending after 1m10.003502875s (10964/s)
2024-01-25T16:08:48.959+0100    INFO    fil-consensus   filcns/upgrades.go:2418 856089 jobs created, 855082 done, 1007 pending after 1m20.003574541s (10688/s)
2024-01-25T16:08:58.964+0100    INFO    fil-consensus   filcns/upgrades.go:2418 960945 jobs created, 959938 done, 1007 pending after 1m30.00758175s (10665/s)
2024-01-25T16:09:08.966+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1073829 jobs created, 1072822 done, 1007 pending after 1m40.009580375s (10727/s)
2024-01-25T16:09:18.966+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1178621 jobs created, 1177801 done, 820 pending after 1m50.0097315s (10706/s)
2024-01-25T16:09:28.966+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1241808 jobs created, 1240801 done, 1007 pending after 2m0.009873s (10339/s)
2024-01-25T16:09:38.966+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1342711 jobs created, 1341704 done, 1007 pending after 2m10.010000666s (10320/s)
2024-01-25T16:09:48.967+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1463172 jobs created, 1462165 done, 1007 pending after 2m20.010071208s (10443/s)
2024-01-25T16:09:58.967+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1582286 jobs created, 1581279 done, 1007 pending after 2m30.010192s (10541/s)
2024-01-25T16:10:08.967+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1687403 jobs created, 1687330 done, 73 pending after 2m40.01027375s (10545/s)
2024-01-25T16:10:18.967+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1828209 jobs created, 1827202 done, 1007 pending after 2m50.010353208s (10748/s)
2024-01-25T16:10:28.968+0100    INFO    fil-consensus   filcns/upgrades.go:2418 1934049 jobs created, 1933042 done, 1007 pending after 3m0.01066725s (10738/s)
2024-01-25T16:10:38.969+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2052296 jobs created, 2051289 done, 1007 pending after 3m10.012419875s (10796/s)
2024-01-25T16:10:48.970+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2171952 jobs created, 2170945 done, 1007 pending after 3m20.012530958s (10854/s)
2024-01-25T16:10:58.970+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2260316 jobs created, 2259309 done, 1007 pending after 3m30.012603625s (10758/s)
2024-01-25T16:11:08.971+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2338685 jobs created, 2338480 done, 205 pending after 3m40.013232708s (10629/s)
2024-01-25T16:11:18.971+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2463665 jobs created, 2462748 done, 917 pending after 3m50.013339625s (10707/s)
2024-01-25T16:11:28.974+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2598835 jobs created, 2597828 done, 1007 pending after 4m0.016053958s (10824/s)
2024-01-25T16:11:38.976+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2729130 jobs created, 2728123 done, 1007 pending after 4m10.018020625s (10912/s)
2024-01-25T16:11:48.976+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2803049 jobs created, 2802042 done, 1007 pending after 4m20.018111s (10776/s)
2024-01-25T16:11:58.976+0100    INFO    fil-consensus   filcns/upgrades.go:2418 2904374 jobs created, 2903367 done, 1007 pending after 4m30.0182235s (10752/s)
2024-01-25T16:12:02.121+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Done creating 2947604 migration jobs after 4m33.163017958s
2024-01-25T16:12:02.121+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 0 done
2024-01-25T16:12:02.469+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 3 done
2024-01-25T16:12:02.480+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 6 done
2024-01-25T16:12:02.618+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 2 done
2024-01-25T16:12:02.960+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 5 done
2024-01-25T16:12:03.072+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 4 done
2024-01-25T16:12:03.287+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Worker 1 done
2024-01-25T16:12:03.287+0100    INFO    fil-consensus   filcns/upgrades.go:2418 All workers done after 4m34.3291245s
2024-01-25T16:12:03.287+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Result writer wrote 2947603 results to state tree after 4m34.329144208s
2024-01-25T16:12:03.287+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Running deferred migrations
2024-01-25T16:12:03.287+0100    INFO    fil-consensus   filcns/upgrades.go:2418 Running deferred migration for f05
2024-01-25T16:14:38.703+0100    INFO    fil-consensus   filcns/upgrades.go:2418 All 2947604 done after 7m9.743619s (6859/s)
2024-01-25T16:14:40.475+0100    ERROR   lotus-shed      lotus-shed/main.go:160  got different results with and without the cache: bafy2bzacecbl6cpzfynizqpoddnunz4pikd5ikdm7jxi3cipbm56jrkuio5sa, bafy2bzacedbolm3bvanxqy7b2n5xxovg2eel3ii35qlfunf2iktkteucd5nb2:
    main.glob..func97
        /Users/magik6k/lotus/cmd/lotus-shed/migrations.go:219

Checklist

Before you mark the PR ready for review, please make sure that:

  • Commits have a clear commit message.
  • PR title is in the form of of <PR type>: <area>: <change being made>
    • example: fix: mempool: Introduce a cache for valid signatures
    • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
    • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
  • If the PR affects users (e.g., new feature, bug fix, system requirements change), update the CHANGELOG.md and add details to the UNRELEASED section.
  • New features have usage guidelines and / or documentation updates in
  • Tests exist for new functionality or change in behavior
  • CI is green

@magik6k magik6k force-pushed the feat/nv22-migration-bench branch 2 times, most recently from 00a73ae to f0c3571 Compare January 29, 2024 12:12
@magik6k magik6k force-pushed the feat/nv22-migration-bench branch from f0c3571 to 1ca66c3 Compare January 29, 2024 12:43
@Reiers
Copy link

Reiers commented Jan 29, 2024

~/lotus$ ./lotus-shed migrate-state --export-bad-migration=migmm.car --repo=/mnt/Lotus_Friday/lotus_test_node 22 bafy2bzacebcf7glgi5r5ng776rcsagn7rderyuwlzo4iv3edcjbxdu6drrdnw
REMINDER: If you are running this, you likely want to ALSO run the continuity testing tool!
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-01-29T14:26:03.522+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Started 125 workers
2024-01-29T14:26:03.522+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-01-29T14:26:03.522+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-01-29T14:26:13.523+0100	INFO	fil-consensus	filcns/upgrades.go:2418	148912 jobs created, 148828 done, 84 pending after 10.000938286s (14881/s)
2024-01-29T14:26:23.523+0100	INFO	fil-consensus	filcns/upgrades.go:2418	237009 jobs created, 236943 done, 66 pending after 20.001680465s (11846/s)
2024-01-29T14:26:33.523+0100	INFO	fil-consensus	filcns/upgrades.go:2418	351829 jobs created, 351759 done, 70 pending after 30.001795165s (11725/s)
2024-01-29T14:26:43.524+0100	INFO	fil-consensus	filcns/upgrades.go:2418	537760 jobs created, 537680 done, 80 pending after 40.001966452s (13441/s)
2024-01-29T14:26:53.524+0100	INFO	fil-consensus	filcns/upgrades.go:2418	708037 jobs created, 707379 done, 658 pending after 50.00210058s (14147/s)
2024-01-29T14:27:03.524+0100	INFO	fil-consensus	filcns/upgrades.go:2418	860783 jobs created, 860706 done, 77 pending after 1m0.002360885s (14345/s)
2024-01-29T14:27:13.524+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1039820 jobs created, 1039737 done, 83 pending after 1m10.002788968s (14853/s)
2024-01-29T14:27:23.525+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1199200 jobs created, 1198075 done, 1125 pending after 1m20.00292573s (14975/s)
2024-01-29T14:27:33.525+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1242728 jobs created, 1242626 done, 102 pending after 1m30.003087136s (13806/s)
2024-01-29T14:27:43.525+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1297086 jobs created, 1297019 done, 67 pending after 1m40.003348111s (12970/s)
2024-01-29T14:27:53.525+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1403393 jobs created, 1403338 done, 55 pending after 1m50.003846916s (12757/s)
2024-01-29T14:28:03.526+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1486771 jobs created, 1486717 done, 54 pending after 2m0.004177029s (12389/s)
2024-01-29T14:28:13.526+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1586722 jobs created, 1586650 done, 72 pending after 2m10.004398466s (12205/s)
2024-01-29T14:28:23.526+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1805845 jobs created, 1805749 done, 96 pending after 2m20.00455059s (12898/s)
2024-01-29T14:28:33.526+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1964486 jobs created, 1964400 done, 86 pending after 2m30.004702653s (13096/s)
2024-01-29T14:28:43.526+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2134783 jobs created, 2134686 done, 97 pending after 2m40.004862999s (13341/s)
2024-01-29T14:28:53.527+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2287625 jobs created, 2286500 done, 1125 pending after 2m50.005796025s (13450/s)
2024-01-29T14:29:03.528+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2340312 jobs created, 2339187 done, 1125 pending after 3m0.006027462s (12995/s)
2024-01-29T14:29:13.594+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2389518 jobs created, 2389322 done, 196 pending after 3m10.006893547s (12575/s)
2024-01-29T14:29:23.594+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2482405 jobs created, 2482305 done, 100 pending after 3m20.072288132s (12407/s)
2024-01-29T14:29:33.594+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2667842 jobs created, 2667742 done, 100 pending after 3m30.072472792s (12699/s)
2024-01-29T14:29:43.594+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2868451 jobs created, 2868362 done, 89 pending after 3m40.072673351s (13034/s)
2024-01-29T14:29:49.129+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2948363 migration jobs after 3m45.60761249s
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 114 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 43 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 96 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 97 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 117 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 99 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 58 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 72 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 79 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 81 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 112 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 39 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 83 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 116 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 62 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-01-29T14:29:49.130+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 120 done
2024-01-29T14:29:49.192+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 104 done
2024-01-29T14:29:49.226+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 38 done
2024-01-29T14:29:49.270+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-01-29T14:29:49.297+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 46 done
2024-01-29T14:29:49.302+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 57 done
2024-01-29T14:29:49.326+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 71 done
2024-01-29T14:29:49.346+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 88 done
2024-01-29T14:29:49.378+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 90 done
2024-01-29T14:29:49.415+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 64 done
2024-01-29T14:29:49.421+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 80 done
2024-01-29T14:29:49.425+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 86 done
2024-01-29T14:29:49.426+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-01-29T14:29:49.433+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-01-29T14:29:49.439+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-01-29T14:29:49.504+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-01-29T14:29:49.512+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-01-29T14:29:49.542+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 73 done
2024-01-29T14:29:49.546+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 74 done
2024-01-29T14:29:49.547+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 103 done
2024-01-29T14:29:49.559+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 45 done
2024-01-29T14:29:49.583+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 60 done
2024-01-29T14:29:49.592+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 76 done
2024-01-29T14:29:49.596+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 121 done
2024-01-29T14:29:49.626+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 66 done
2024-01-29T14:29:49.711+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 115 done
2024-01-29T14:29:49.792+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 61 done
2024-01-29T14:29:49.810+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 98 done
2024-01-29T14:29:49.851+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-01-29T14:29:49.868+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 49 done
2024-01-29T14:29:49.902+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 95 done
2024-01-29T14:29:49.920+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 107 done
2024-01-29T14:29:49.978+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-01-29T14:29:49.985+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 65 done
2024-01-29T14:29:49.996+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 63 done
2024-01-29T14:29:50.006+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 68 done
2024-01-29T14:29:50.053+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 118 done
2024-01-29T14:29:50.112+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 54 done
2024-01-29T14:29:50.217+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-01-29T14:29:50.381+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 111 done
2024-01-29T14:29:50.718+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 59 done
2024-01-29T14:29:50.793+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 69 done
2024-01-29T14:29:50.878+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-01-29T14:29:50.944+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-01-29T14:29:50.955+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-01-29T14:29:50.969+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 67 done
2024-01-29T14:29:51.018+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 110 done
2024-01-29T14:29:51.090+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 42 done
2024-01-29T14:29:51.120+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 40 done
2024-01-29T14:29:51.181+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 78 done
2024-01-29T14:29:51.236+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 113 done
2024-01-29T14:29:51.416+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-01-29T14:29:51.489+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 37 done
2024-01-29T14:29:51.537+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 93 done
2024-01-29T14:29:51.653+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 106 done
2024-01-29T14:29:51.736+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 82 done
2024-01-29T14:29:52.263+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-01-29T14:29:52.483+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 91 done
2024-01-29T14:29:52.525+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 84 done
2024-01-29T14:29:52.665+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 53 done
2024-01-29T14:29:52.732+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 122 done
2024-01-29T14:29:52.817+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 75 done
2024-01-29T14:29:52.923+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-01-29T14:29:52.946+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 94 done
2024-01-29T14:29:52.971+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 108 done
2024-01-29T14:29:52.977+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 124 done
2024-01-29T14:29:52.981+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 123 done
2024-01-29T14:29:53.016+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-01-29T14:29:53.596+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2948363 jobs created, 2948331 done, 32 pending after 3m50.07414467s (12815/s)
2024-01-29T14:29:53.628+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 101 done
2024-01-29T14:29:53.746+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-01-29T14:29:53.795+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-01-29T14:29:53.833+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-01-29T14:29:53.948+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 56 done
2024-01-29T14:29:53.983+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-01-29T14:29:54.092+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-01-29T14:29:54.284+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-01-29T14:29:54.291+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 41 done
2024-01-29T14:29:54.424+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 102 done
2024-01-29T14:29:54.486+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 109 done
2024-01-29T14:29:54.797+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-01-29T14:29:54.855+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 92 done
2024-01-29T14:29:54.912+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 119 done
2024-01-29T14:29:55.071+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 50 done
2024-01-29T14:29:55.163+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-01-29T14:29:55.742+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 100 done
2024-01-29T14:29:55.862+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 55 done
2024-01-29T14:29:55.940+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-01-29T14:29:55.998+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 44 done
2024-01-29T14:29:57.054+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 52 done
2024-01-29T14:29:57.962+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 70 done
2024-01-29T14:29:58.649+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 87 done
2024-01-29T14:29:58.895+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 105 done
2024-01-29T14:29:59.835+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 51 done
2024-01-29T14:30:03.021+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 48 done
2024-01-29T14:30:03.596+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2948363 jobs created, 2948357 done, 6 pending after 4m0.074514753s (12281/s)
2024-01-29T14:30:03.919+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 47 done
2024-01-29T14:30:04.197+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-01-29T14:30:04.510+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-01-29T14:30:09.311+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 77 done
2024-01-29T14:30:12.711+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 89 done
2024-01-29T14:30:13.596+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2948363 jobs created, 2948362 done, 1 pending after 4m10.074643536s (11790/s)
2024-01-29T14:30:13.802+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 85 done
2024-01-29T14:30:13.802+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2948362 results to state tree after 4m10.280428929s
2024-01-29T14:30:13.802+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 4m10.280396117s
2024-01-29T14:30:13.802+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-01-29T14:30:13.803+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-01-29T14:34:06.950+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All 2948363 done after 8m3.428579356s (6099/s)
migration height  3599392
old cid  bafy2bzacedx4ywdol2hgejjzvgibk43szbelqu67lkfyts43ou6dx7b3md776
new cid  bafy2bzaceacmulmiciiekymsnnrxffjqpyvkuffpnyvitbo26lunfjtizf6jg
completed round actual (without cache), took  8m11.351107804s
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-01-29T14:34:15.286+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-01-29T14:34:15.286+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Started 64 workers
2024-01-29T14:34:15.286+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-01-29T14:36:52.766+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2948087 migration jobs after 2m37.47976209s
2024-01-29T14:36:52.767+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-01-29T14:36:52.767+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 58 done
2024-01-29T14:36:52.767+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-01-29T14:36:52.767+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 46 done
2024-01-29T14:36:52.767+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 41 done
2024-01-29T14:36:52.767+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 39 done
2024-01-29T14:36:52.767+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 50 done
2024-01-29T14:36:52.767+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-01-29T14:36:52.802+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-01-29T14:36:52.805+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-01-29T14:36:52.811+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-01-29T14:36:52.867+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-01-29T14:36:53.044+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 40 done
2024-01-29T14:36:53.050+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 60 done
2024-01-29T14:36:53.203+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 45 done
2024-01-29T14:36:53.214+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-01-29T14:36:53.216+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 52 done
2024-01-29T14:36:53.326+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-01-29T14:36:53.339+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 55 done
2024-01-29T14:36:53.342+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 63 done
2024-01-29T14:36:53.455+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 62 done
2024-01-29T14:36:53.506+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 59 done
2024-01-29T14:36:53.507+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-01-29T14:36:53.721+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-01-29T14:36:53.735+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-01-29T14:36:53.803+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-01-29T14:36:53.814+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 51 done
2024-01-29T14:36:53.848+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-01-29T14:36:53.879+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 49 done
2024-01-29T14:36:53.903+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-01-29T14:36:54.029+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-01-29T14:36:54.086+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-01-29T14:36:54.267+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 37 done
2024-01-29T14:36:54.287+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-01-29T14:36:54.387+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-01-29T14:36:54.400+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 42 done
2024-01-29T14:36:54.452+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-01-29T14:36:54.471+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 54 done
2024-01-29T14:36:54.493+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-01-29T14:36:54.557+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 61 done
2024-01-29T14:36:54.565+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-01-29T14:36:54.581+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 43 done
2024-01-29T14:36:54.686+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-01-29T14:36:54.690+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-01-29T14:36:54.737+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 56 done
2024-01-29T14:36:54.753+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-01-29T14:36:54.883+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 57 done
2024-01-29T14:36:54.971+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 47 done
2024-01-29T14:36:55.194+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 38 done
2024-01-29T14:36:55.414+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-01-29T14:36:55.440+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-01-29T14:36:55.613+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-01-29T14:36:55.635+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-01-29T14:36:56.095+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-01-29T14:36:56.097+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-01-29T14:36:56.197+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-01-29T14:36:56.321+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-01-29T14:36:56.390+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 53 done
2024-01-29T14:36:56.592+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-01-29T14:36:58.178+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-01-29T14:36:58.428+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 48 done
2024-01-29T14:36:59.153+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-01-29T14:36:59.162+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-01-29T14:37:01.920+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 44 done
2024-01-29T14:37:01.920+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 2m46.6339633s
2024-01-29T14:37:01.920+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2948086 results to state tree after 2m46.633999658s
2024-01-29T14:37:01.920+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-01-29T14:37:01.920+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-01-29T14:38:26.059+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All 2948087 done after 4m10.773095855s (11756/s)
completed premigration, took  4m13.870492981s
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-01-29T14:38:28.725+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-01-29T14:38:28.725+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Started 125 workers
2024-01-29T14:38:28.725+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-01-29T14:38:38.725+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1586916 jobs created, 1586901 done, 15 pending after 10.000237788s (158686/s)
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2948363 migration jobs after 17.345184347s
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 43 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 73 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 85 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 123 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 63 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 94 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 106 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 64 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 89 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 102 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 80 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 113 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 104 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 115 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 54 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 101 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 37 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 65 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 69 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 88 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 91 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 124 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 78 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 120 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 112 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 84 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 51 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 81 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 79 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 44 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 49 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 95 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 70 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 108 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 48 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 56 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 62 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 98 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 67 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 83 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 58 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 119 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 40 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 118 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 90 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 52 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 55 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 45 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 60 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 87 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 71 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 97 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 109 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 86 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 57 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 103 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 59 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 92 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 74 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 76 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 46 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 99 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 39 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 100 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 53 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 61 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 42 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 38 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 116 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 105 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 75 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 50 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 96 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 68 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 117 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 111 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 72 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 122 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 114 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 66 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 93 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 110 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 121 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 107 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 82 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 47 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 77 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-01-29T14:38:46.070+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 41 done
2024-01-29T14:38:46.071+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2948362 results to state tree after 17.346888992s
2024-01-29T14:38:46.072+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 17.346887649s
2024-01-29T14:38:46.072+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-01-29T14:38:46.072+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
missing deal?? weird market cron? 8869440
missing deal?? weird market cron? 8869441
missing deal?? weird market cron? 8869442
missing deal?? weird market cron? 8869443
missing deal?? weird market cron? 9128334
missing deal?? weird market cron? 9214911
missing deal?? weird market cron? 9387337
missing deal?? weird market cron? 9560510
missing deal?? weird market cron? 10164979
missing deal?? weird market cron? 10164980
missing deal?? weird market cron? 10164981
missing deal?? weird market cron? 10164982
missing deal?? weird market cron? 10164983
missing deal?? weird market cron? 10164984
missing deal?? weird market cron? 10164985
missing deal?? weird market cron? 10164986
missing deal?? weird market cron? 10165128
missing deal?? weird market cron? 10165129
missing deal?? weird market cron? 10165131
missing deal?? weird market cron? 10165133
missing deal?? weird market cron? 10856343
missing deal?? weird market cron? 10856344
missing deal?? weird market cron? 10856345
missing deal?? weird market cron? 10856346
missing deal?? weird market cron? 10856347
missing deal?? weird market cron? 10856348
missing deal?? weird market cron? 10856349
missing deal?? weird market cron? 10856350
missing deal?? weird market cron? 11375008
missing deal?? weird market cron? 11375009
missing deal?? weird market cron? 11375010
missing deal?? weird market cron? 11375011
missing deal?? weird market cron? 11375012
missing deal?? weird market cron? 11375013
missing deal?? weird market cron? 11375014
missing deal?? weird market cron? 11375015
missing deal?? weird market cron? 11461101
missing deal?? weird market cron? 11461102
missing deal?? weird market cron? 11461103
missing deal?? weird market cron? 11461104
missing deal?? weird market cron? 11461105
missing deal?? weird market cron? 11461106
missing deal?? weird market cron? 11461107
missing deal?? weird market cron? 11461108
missing deal?? weird market cron? 11893264
missing deal?? weird market cron? 11893265
missing deal?? weird market cron? 11893266
missing deal?? weird market cron? 11893267
missing deal?? weird market cron? 11893268
missing deal?? weird market cron? 11893269
missing deal?? weird market cron? 11893270
missing deal?? weird market cron? 11893271
missing deal?? weird market cron? 12152319
missing deal?? weird market cron? 12152320
missing deal?? weird market cron? 12152321
missing deal?? weird market cron? 12152322
missing deal?? weird market cron? 12152323
missing deal?? weird market cron? 12152324
missing deal?? weird market cron? 12152325
missing deal?? weird market cron? 12152326
missing deal?? weird market cron? 12239272
missing deal?? weird market cron? 12497947
missing deal?? weird market cron? 12497948
missing deal?? weird market cron? 18287036
missing deal?? weird market cron? 20014956
missing deal?? weird market cron? 20014957
missing deal?? weird market cron? 20014958
missing deal?? weird market cron? 20014959
missing deal?? weird market cron? 20014960
missing deal?? weird market cron? 20014961
missing deal?? weird market cron? 20014962
missing deal?? weird market cron? 20014963
missing deal?? weird market cron? 20187257
missing deal?? weird market cron? 20187258
missing deal?? weird market cron? 20187259
missing deal?? weird market cron? 20187260
missing deal?? weird market cron? 20187261
missing deal?? weird market cron? 20187262
missing deal?? weird market cron? 20187263
missing deal?? weird market cron? 20187264
missing deal?? weird market cron? 20274405
missing deal?? weird market cron? 20274406
missing deal?? weird market cron? 20274407
missing deal?? weird market cron? 20274408
missing deal?? weird market cron? 20274409
missing deal?? weird market cron? 20274410
missing deal?? weird market cron? 20274411
missing deal?? weird market cron? 20274412
missing deal?? weird market cron? 31506546
missing deal?? weird market cron? 32715669
missing deal?? weird market cron? 36344640
missing deal?? weird market cron? 36344650
missing deal?? weird market cron? 47231294
missing deal?? weird market cron? 47231298
missing deal?? weird market cron? 47835522
missing deal?? weird market cron? 47835523
missing deal?? weird market cron? 47835524
missing deal?? weird market cron? 47835525
missing deal?? weird market cron? 47835526
missing deal?? weird market cron? 47835527
missing deal?? weird market cron? 47835528
missing deal?? weird market cron? 47835529
missing deal?? weird market cron? 58030675
missing deal?? weird market cron? 58031309
missing deal?? weird market cron? 67621691
missing deal?? weird market cron? 67621692
missing deal?? weird market cron? 67621693
missing deal?? weird market cron? 67621694
missing deal?? weird market cron? 68571277
missing deal?? weird market cron? 68571280
missing deal?? weird market cron? 68571281
missing deal?? weird market cron? 68744074
missing deal?? weird market cron? 68744080
2024-01-29T14:38:48.608+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All 2948363 done after 19.88344357s (148282/s)
completed round actual (with cache), took  22.30257515s

@rjan90
Copy link
Contributor

rjan90 commented Jan 29, 2024

./lotus-shed migrate-state --repo=/mnt/lotuschain/.lotus 22 bafy2bzaced2om4h63hn5ividakr62buxdsw7pd4latwustz7lpfannoozlw4e
REMINDER: If you are running this, you likely want to ALSO run the continuity testing tool!
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-01-29T14:26:29.789+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Started 37 workers
2024-01-29T14:26:29.789+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-01-29T14:26:29.789+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-01-29T14:26:39.789+0100	INFO	fil-consensus	filcns/upgrades.go:2418	119062 jobs created, 118025 done, 1037 pending after 10.000259081s (11802/s)
2024-01-29T14:26:49.789+0100	INFO	fil-consensus	filcns/upgrades.go:2418	217115 jobs created, 216078 done, 1037 pending after 20.000604869s (10804/s)
2024-01-29T14:26:59.790+0100	INFO	fil-consensus	filcns/upgrades.go:2418	318200 jobs created, 317163 done, 1037 pending after 30.000828822s (10572/s)
2024-01-29T14:27:09.790+0100	INFO	fil-consensus	filcns/upgrades.go:2418	422953 jobs created, 421926 done, 1027 pending after 40.001011265s (10548/s)
2024-01-29T14:27:19.794+0100	INFO	fil-consensus	filcns/upgrades.go:2418	541100 jobs created, 540063 done, 1037 pending after 50.005578917s (10800/s)
2024-01-29T14:27:29.795+0100	INFO	fil-consensus	filcns/upgrades.go:2418	609001 jobs created, 608281 done, 720 pending after 1m0.005919491s (10137/s)
2024-01-29T14:27:39.795+0100	INFO	fil-consensus	filcns/upgrades.go:2418	714002 jobs created, 712973 done, 1029 pending after 1m10.006144525s (10184/s)
2024-01-29T14:27:49.795+0100	INFO	fil-consensus	filcns/upgrades.go:2418	802835 jobs created, 801798 done, 1037 pending after 1m20.006471854s (10022/s)
2024-01-29T14:27:59.796+0100	INFO	fil-consensus	filcns/upgrades.go:2418	885458 jobs created, 884421 done, 1037 pending after 1m30.006969551s (9826/s)
2024-01-29T14:28:09.796+0100	INFO	fil-consensus	filcns/upgrades.go:2418	971973 jobs created, 970936 done, 1037 pending after 1m40.007449129s (9709/s)
2024-01-29T14:28:19.796+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1062065 jobs created, 1061028 done, 1037 pending after 1m50.007677065s (9645/s)
2024-01-29T14:28:29.797+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1133819 jobs created, 1132797 done, 1022 pending after 2m0.008319826s (9439/s)
2024-01-29T14:28:39.797+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1189136 jobs created, 1188099 done, 1037 pending after 2m10.008533701s (9139/s)
2024-01-29T14:28:49.798+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1204673 jobs created, 1203636 done, 1037 pending after 2m20.00918166s (8597/s)
2024-01-29T14:28:59.800+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1275415 jobs created, 1274378 done, 1037 pending after 2m30.011039614s (8495/s)
2024-01-29T14:29:09.800+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1399622 jobs created, 1398585 done, 1037 pending after 2m40.011206616s (8741/s)
2024-01-29T14:29:19.800+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1515449 jobs created, 1514412 done, 1037 pending after 2m50.011376249s (8908/s)
2024-01-29T14:29:29.800+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1612403 jobs created, 1611771 done, 632 pending after 3m0.011712839s (8954/s)
2024-01-29T14:29:39.801+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1719314 jobs created, 1718277 done, 1037 pending after 3m10.011948096s (9043/s)
2024-01-29T14:29:49.801+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1819883 jobs created, 1818846 done, 1037 pending after 3m20.012197019s (9094/s)
2024-01-29T14:29:59.802+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1895253 jobs created, 1894237 done, 1016 pending after 3m30.012955184s (9020/s)
2024-01-29T14:30:09.802+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1989114 jobs created, 1988116 done, 998 pending after 3m40.013134073s (9036/s)
2024-01-29T14:30:19.802+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2060332 jobs created, 2059295 done, 1037 pending after 3m50.013376053s (8953/s)
2024-01-29T14:30:29.803+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2145842 jobs created, 2144805 done, 1037 pending after 4m0.014079586s (8936/s)
2024-01-29T14:30:39.803+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2216711 jobs created, 2215674 done, 1037 pending after 4m10.014683959s (8862/s)
2024-01-29T14:30:49.805+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2283176 jobs created, 2282139 done, 1037 pending after 4m20.016171019s (8777/s)
2024-01-29T14:30:59.808+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2361406 jobs created, 2360369 done, 1037 pending after 4m30.016456153s (8742/s)
2024-01-29T14:31:09.808+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2377730 jobs created, 2376693 done, 1037 pending after 4m40.019295783s (8488/s)
2024-01-29T14:31:19.809+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2385594 jobs created, 2384557 done, 1037 pending after 4m50.020013746s (8222/s)
2024-01-29T14:31:29.809+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2421362 jobs created, 2420325 done, 1037 pending after 5m0.020188136s (8067/s)
2024-01-29T14:31:39.809+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2465171 jobs created, 2464134 done, 1037 pending after 5m10.020356625s (7948/s)
2024-01-29T14:31:49.809+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2520144 jobs created, 2519107 done, 1037 pending after 5m20.020514819s (7872/s)
2024-01-29T14:31:59.809+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2565467 jobs created, 2564464 done, 1003 pending after 5m30.020677248s (7771/s)
2024-01-29T14:32:09.810+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2638583 jobs created, 2637546 done, 1037 pending after 5m40.020877352s (7757/s)
2024-01-29T14:32:19.810+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2748777 jobs created, 2747793 done, 984 pending after 5m50.02109945s (7850/s)
2024-01-29T14:32:29.810+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2829982 jobs created, 2828946 done, 1036 pending after 6m0.021250765s (7858/s)
2024-01-29T14:32:39.811+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2913951 jobs created, 2912918 done, 1033 pending after 6m10.021992528s (7872/s)
2024-01-29T14:32:43.634+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2950188 migration jobs after 6m13.845553291s
2024-01-29T14:32:43.708+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-01-29T14:32:43.708+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-01-29T14:32:43.916+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-01-29T14:32:44.009+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-01-29T14:32:44.101+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-01-29T14:32:44.103+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-01-29T14:32:44.144+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-01-29T14:32:44.205+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-01-29T14:32:44.256+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-01-29T14:32:44.333+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-01-29T14:32:44.341+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-01-29T14:32:44.489+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-01-29T14:32:44.722+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-01-29T14:32:44.829+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-01-29T14:32:44.841+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-01-29T14:32:44.938+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-01-29T14:32:45.095+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-01-29T14:32:46.007+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-01-29T14:32:46.083+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-01-29T14:32:46.310+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-01-29T14:32:46.350+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-01-29T14:32:46.383+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-01-29T14:32:46.387+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-01-29T14:32:46.449+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-01-29T14:32:46.906+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-01-29T14:32:48.059+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-01-29T14:32:48.563+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-01-29T14:32:48.567+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-01-29T14:32:48.777+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-01-29T14:32:49.135+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-01-29T14:32:49.287+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-01-29T14:32:49.726+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-01-29T14:32:49.811+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2950188 jobs created, 2950183 done, 5 pending after 6m20.022196031s (7763/s)
2024-01-29T14:32:49.892+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-01-29T14:32:50.106+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-01-29T14:32:52.571+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-01-29T14:32:53.957+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-01-29T14:32:59.811+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2950188 jobs created, 2950187 done, 1 pending after 6m30.022340855s (7564/s)
2024-01-29T14:33:00.616+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-01-29T14:33:00.616+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 6m30.827032077s
2024-01-29T14:33:00.616+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2950187 results to state tree after 6m30.827077208s
2024-01-29T14:33:00.616+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-01-29T14:33:00.616+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-01-29T14:37:07.067+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All 2950188 done after 10m37.27833309s (4629/s)
migration height  3607608
old cid  bafy2bzacearxqnq6ppd6w2uqri4un3eol3u3c2qvkmdfcmjokl4roi3yx37js
new cid  bafy2bzaceanctncjt6bmbsfvetzewnnihtpbg5r62bhnoewhtz55dsghyteoy
completed round actual (without cache), took  10m55.905387915s
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-01-29T14:37:26.139+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Started 20 workers
2024-01-29T14:37:26.139+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-01-29T14:37:26.139+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-01-29T14:42:26.139+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2292987 jobs created, 2292967 done, 20 pending after 5m0.000268085s (7643/s)
2024-01-29T14:44:13.023+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2949864 migration jobs after 6m46.876365225s
2024-01-29T14:44:13.023+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-01-29T14:44:13.023+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-01-29T14:44:13.023+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-01-29T14:44:13.048+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-01-29T14:44:13.374+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-01-29T14:44:13.512+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-01-29T14:44:13.533+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-01-29T14:44:13.549+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-01-29T14:44:13.592+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-01-29T14:44:13.787+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-01-29T14:44:13.858+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-01-29T14:44:14.433+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-01-29T14:44:14.745+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-01-29T14:44:14.779+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-01-29T14:44:15.607+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-01-29T14:44:15.675+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-01-29T14:44:15.694+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-01-29T14:44:16.057+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-01-29T14:44:16.547+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-01-29T14:44:20.298+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-01-29T14:44:20.305+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2949863 results to state tree after 6m54.159598816s
2024-01-29T14:44:20.306+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 6m54.159551262s
2024-01-29T14:44:20.307+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-01-29T14:44:20.307+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-01-29T14:46:39.251+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All 2949864 done after 9m13.11191334s (5333/s)
completed premigration, took  9m18.333504207s
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-01-29T14:46:43.988+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-01-29T14:46:43.988+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Started 37 workers
2024-01-29T14:46:43.989+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-01-29T14:46:53.989+0100	INFO	fil-consensus	filcns/upgrades.go:2418	573944 jobs created, 573941 done, 3 pending after 10.000765034s (57390/s)
2024-01-29T14:47:03.989+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1306704 jobs created, 1306703 done, 1 pending after 20.001003889s (65332/s)
2024-01-29T14:47:13.989+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2064387 jobs created, 2064297 done, 90 pending after 30.001192356s (68807/s)
2024-01-29T14:47:23.990+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2923454 jobs created, 2923454 done, 0 pending after 40.001444721s (73084/s)
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2950188 migration jobs after 40.349361472s
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-01-29T14:47:24.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-01-29T14:47:24.339+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 40.350520588s
2024-01-29T14:47:24.339+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2950187 results to state tree after 40.350553334s
2024-01-29T14:47:24.339+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-01-29T14:47:24.339+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
missing deal?? weird market cron? 9568480
missing deal?? weird market cron? 9568481
missing deal?? weird market cron? 9568482
missing deal?? weird market cron? 9568483
missing deal?? weird market cron? 9568484
missing deal?? weird market cron? 9568485
missing deal?? weird market cron? 9568486
missing deal?? weird market cron? 9568487
missing deal?? weird market cron? 10087550
missing deal?? weird market cron? 10087551
missing deal?? weird market cron? 10087552
missing deal?? weird market cron? 10259955
missing deal?? weird market cron? 10259956
missing deal?? weird market cron? 10259957
missing deal?? weird market cron? 10259958
missing deal?? weird market cron? 11037073
missing deal?? weird market cron? 11037074
missing deal?? weird market cron? 11037075
missing deal?? weird market cron? 11037076
missing deal?? weird market cron? 11037077
missing deal?? weird market cron? 11296809
missing deal?? weird market cron? 11296814
missing deal?? weird market cron? 11383063
missing deal?? weird market cron? 11383064
missing deal?? weird market cron? 11383065
missing deal?? weird market cron? 11383066
missing deal?? weird market cron? 11383067
missing deal?? weird market cron? 11383068
missing deal?? weird market cron? 11383069
missing deal?? weird market cron? 11383070
missing deal?? weird market cron? 11901969
missing deal?? weird market cron? 11901970
missing deal?? weird market cron? 11901971
missing deal?? weird market cron? 11901972
missing deal?? weird market cron? 11901973
missing deal?? weird market cron? 11901974
missing deal?? weird market cron? 11901975
missing deal?? weird market cron? 11901976
missing deal?? weird market cron? 12074487
missing deal?? weird market cron? 12074488
missing deal?? weird market cron? 12160598
missing deal?? weird market cron? 12160599
missing deal?? weird market cron? 12160600
missing deal?? weird market cron? 12160601
missing deal?? weird market cron? 12160602
missing deal?? weird market cron? 12160603
missing deal?? weird market cron? 12333230
missing deal?? weird market cron? 12419947
missing deal?? weird market cron? 12419948
missing deal?? weird market cron? 12419949
missing deal?? weird market cron? 12419950
missing deal?? weird market cron? 12419951
missing deal?? weird market cron? 12419952
missing deal?? weird market cron? 12419953
missing deal?? weird market cron? 12419954
missing deal?? weird market cron? 12678961
missing deal?? weird market cron? 14061147
missing deal?? weird market cron? 14061149
missing deal?? weird market cron? 14148280
missing deal?? weird market cron? 14406852
missing deal?? weird market cron? 14406853
missing deal?? weird market cron? 14406854
missing deal?? weird market cron? 14406855
missing deal?? weird market cron? 14579993
missing deal?? weird market cron? 14579994
missing deal?? weird market cron? 14579995
missing deal?? weird market cron? 14579996
missing deal?? weird market cron? 14579997
missing deal?? weird market cron? 14579998
missing deal?? weird market cron? 14579999
missing deal?? weird market cron? 14925332
missing deal?? weird market cron? 15875724
missing deal?? weird market cron? 16048563
missing deal?? weird market cron? 16048566
missing deal?? weird market cron? 16566806
missing deal?? weird market cron? 16826657
missing deal?? weird market cron? 17517939
missing deal?? weird market cron? 17517940
missing deal?? weird market cron? 17517941
missing deal?? weird market cron? 18208371
missing deal?? weird market cron? 18208372
missing deal?? weird market cron? 18208373
missing deal?? weird market cron? 18208374
missing deal?? weird market cron? 18468350
missing deal?? weird market cron? 20023180
missing deal?? weird market cron? 20023181
missing deal?? weird market cron? 20023182
missing deal?? weird market cron? 20023183
missing deal?? weird market cron? 20023184
missing deal?? weird market cron? 20023185
missing deal?? weird market cron? 20023186
missing deal?? weird market cron? 20023187
missing deal?? weird market cron? 31686734
missing deal?? weird market cron? 31686738
missing deal?? weird market cron? 42055473
missing deal?? weird market cron? 42055485
missing deal?? weird market cron? 42055558
missing deal?? weird market cron? 42055559
missing deal?? weird market cron? 42055560
missing deal?? weird market cron? 42055561
missing deal?? weird market cron? 42055562
missing deal?? weird market cron? 42055563
missing deal?? weird market cron? 42055564
missing deal?? weird market cron? 42055565
missing deal?? weird market cron? 42055566
missing deal?? weird market cron? 42055567
missing deal?? weird market cron? 42055568
missing deal?? weird market cron? 42055569
missing deal?? weird market cron? 42055570
missing deal?? weird market cron? 42055571
missing deal?? weird market cron? 42055572
missing deal?? weird market cron? 42055573
missing deal?? weird market cron? 42055574
missing deal?? weird market cron? 42055575
missing deal?? weird market cron? 42055576
missing deal?? weird market cron? 42055577
missing deal?? weird market cron? 42055578
missing deal?? weird market cron? 42055579
missing deal?? weird market cron? 42055580
missing deal?? weird market cron? 42055581
missing deal?? weird market cron? 42055582
missing deal?? weird market cron? 42400270
missing deal?? weird market cron? 44042614
missing deal?? weird market cron? 44042627
missing deal?? weird market cron? 47843555
missing deal?? weird market cron? 47843556
missing deal?? weird market cron? 47843557
missing deal?? weird market cron? 47843558
missing deal?? weird market cron? 47843559
missing deal?? weird market cron? 47843560
missing deal?? weird market cron? 47843561
missing deal?? weird market cron? 48621481
missing deal?? weird market cron? 48621482
missing deal?? weird market cron? 48621483
missing deal?? weird market cron? 48621484
missing deal?? weird market cron? 48621485
missing deal?? weird market cron? 48621486
missing deal?? weird market cron? 48621487
missing deal?? weird market cron? 48621488
missing deal?? weird market cron? 52682022
missing deal?? weird market cron? 67110758
missing deal?? weird market cron? 67110780
missing deal?? weird market cron? 67110895
missing deal?? weird market cron? 67110897
missing deal?? weird market cron? 67110906
missing deal?? weird market cron? 67110927
missing deal?? weird market cron? 67110937
missing deal?? weird market cron? 67110982
missing deal?? weird market cron? 67111604
missing deal?? weird market cron? 67111607
2024-01-29T14:47:29.335+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All 2950188 done after 45.347109594s (65058/s)
completed round actual (with cache), took  50.549088812s

@TippyFlitsUK
Copy link
Contributor

./lotus-shed migrate-state --repo=/filecoin-sealing/.lotus-daemon 22 bafy2bzaced2om4h63hn5ividakr62buxdsw7pd4latwustz7lpfannoozlw4e
REMINDER: If you are running this, you likely want to ALSO run the continuity testing tool!
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-01-29T23:56:29.201Z	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-01-29T23:56:29.201Z	INFO	fil-consensus	filcns/upgrades.go:2418	Started 61 workers
2024-01-29T23:56:29.202Z	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-01-29T23:56:39.202Z	INFO	fil-consensus	filcns/upgrades.go:2418	152330 jobs created, 151269 done, 1061 pending after 10.000497272s (15126/s)
2024-01-29T23:56:49.202Z	INFO	fil-consensus	filcns/upgrades.go:2418	260583 jobs created, 259525 done, 1058 pending after 20.001123814s (12976/s)
2024-01-29T23:56:59.203Z	INFO	fil-consensus	filcns/upgrades.go:2418	380286 jobs created, 379229 done, 1057 pending after 30.00155242s (12640/s)
2024-01-29T23:57:09.203Z	INFO	fil-consensus	filcns/upgrades.go:2418	488272 jobs created, 487211 done, 1061 pending after 40.001850141s (12180/s)
2024-01-29T23:57:19.204Z	INFO	fil-consensus	filcns/upgrades.go:2418	599251 jobs created, 598203 done, 1048 pending after 50.002358264s (11963/s)
2024-01-29T23:57:29.204Z	INFO	fil-consensus	filcns/upgrades.go:2418	674435 jobs created, 673374 done, 1061 pending after 1m0.002687028s (11222/s)
2024-01-29T23:57:39.204Z	INFO	fil-consensus	filcns/upgrades.go:2418	803628 jobs created, 802567 done, 1061 pending after 1m10.003129072s (11465/s)
2024-01-29T23:57:49.205Z	INFO	fil-consensus	filcns/upgrades.go:2418	897597 jobs created, 897016 done, 581 pending after 1m20.00387771s (11212/s)
2024-01-29T23:57:59.206Z	INFO	fil-consensus	filcns/upgrades.go:2418	990449 jobs created, 989388 done, 1061 pending after 1m30.00478483s (10993/s)
2024-01-29T23:58:09.206Z	INFO	fil-consensus	filcns/upgrades.go:2418	1078603 jobs created, 1077542 done, 1061 pending after 1m40.005162828s (10775/s)
2024-01-29T23:58:19.207Z	INFO	fil-consensus	filcns/upgrades.go:2418	1176171 jobs created, 1175110 done, 1061 pending after 1m50.005675045s (10682/s)
2024-01-29T23:58:29.207Z	INFO	fil-consensus	filcns/upgrades.go:2418	1222436 jobs created, 1221375 done, 1061 pending after 2m0.00590187s (10178/s)
2024-01-29T23:58:39.207Z	INFO	fil-consensus	filcns/upgrades.go:2418	1267192 jobs created, 1266131 done, 1061 pending after 2m10.006215694s (9739/s)
2024-01-29T23:58:49.208Z	INFO	fil-consensus	filcns/upgrades.go:2418	1349215 jobs created, 1348412 done, 803 pending after 2m20.006414676s (9631/s)
2024-01-29T23:58:59.209Z	INFO	fil-consensus	filcns/upgrades.go:2418	1487390 jobs created, 1487332 done, 58 pending after 2m30.007265212s (9915/s)
2024-01-29T23:59:09.209Z	INFO	fil-consensus	filcns/upgrades.go:2418	1623879 jobs created, 1622818 done, 1061 pending after 2m40.007610294s (10142/s)
2024-01-29T23:59:19.210Z	INFO	fil-consensus	filcns/upgrades.go:2418	1750131 jobs created, 1749070 done, 1061 pending after 2m50.008468588s (10288/s)
2024-01-29T23:59:29.210Z	INFO	fil-consensus	filcns/upgrades.go:2418	1876068 jobs created, 1875007 done, 1061 pending after 3m0.00896369s (10416/s)
2024-01-29T23:59:39.220Z	INFO	fil-consensus	filcns/upgrades.go:2418	1971843 jobs created, 1970782 done, 1061 pending after 3m10.018651357s (10372/s)
2024-01-29T23:59:49.220Z	INFO	fil-consensus	filcns/upgrades.go:2418	2089652 jobs created, 2088591 done, 1061 pending after 3m20.019158297s (10442/s)
2024-01-29T23:59:59.221Z	INFO	fil-consensus	filcns/upgrades.go:2418	2189075 jobs created, 2188014 done, 1061 pending after 3m30.019517885s (10418/s)
2024-01-30T00:00:09.221Z	INFO	fil-consensus	filcns/upgrades.go:2418	2278748 jobs created, 2277715 done, 1033 pending after 3m40.020033653s (10352/s)
2024-01-30T00:00:19.226Z	INFO	fil-consensus	filcns/upgrades.go:2418	2380870 jobs created, 2379809 done, 1061 pending after 3m50.024363055s (10346/s)
2024-01-30T00:00:29.226Z	INFO	fil-consensus	filcns/upgrades.go:2418	2429616 jobs created, 2428555 done, 1061 pending after 4m0.024660381s (10118/s)
2024-01-30T00:00:39.226Z	INFO	fil-consensus	filcns/upgrades.go:2418	2455585 jobs created, 2454524 done, 1061 pending after 4m10.024995375s (9817/s)
2024-01-30T00:00:49.227Z	INFO	fil-consensus	filcns/upgrades.go:2418	2512246 jobs created, 2511185 done, 1061 pending after 4m20.025240389s (9657/s)
2024-01-30T00:00:59.227Z	INFO	fil-consensus	filcns/upgrades.go:2418	2576525 jobs created, 2575464 done, 1061 pending after 4m30.025761423s (9538/s)
2024-01-30T00:01:09.228Z	INFO	fil-consensus	filcns/upgrades.go:2418	2667081 jobs created, 2666020 done, 1061 pending after 4m40.026627059s (9521/s)
2024-01-30T00:01:19.228Z	INFO	fil-consensus	filcns/upgrades.go:2418	2781280 jobs created, 2780219 done, 1061 pending after 4m50.02684822s (9586/s)
2024-01-30T00:01:29.228Z	INFO	fil-consensus	filcns/upgrades.go:2418	2873439 jobs created, 2872378 done, 1061 pending after 5m0.027138721s (9574/s)
2024-01-30T00:01:37.317Z	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2950188 migration jobs after 5m8.115593382s
2024-01-30T00:01:37.351Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-01-30T00:01:37.351Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-01-30T00:01:37.351Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-01-30T00:01:37.351Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 41 done
2024-01-30T00:01:37.351Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 45 done
2024-01-30T00:01:37.351Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-01-30T00:01:37.358Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-01-30T00:01:37.372Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-01-30T00:01:37.375Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-01-30T00:01:37.427Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-01-30T00:01:37.474Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-01-30T00:01:37.521Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-01-30T00:01:37.628Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-01-30T00:01:37.704Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-01-30T00:01:37.760Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-01-30T00:01:37.821Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 52 done
2024-01-30T00:01:37.829Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-01-30T00:01:37.857Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 56 done
2024-01-30T00:01:37.901Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 53 done
2024-01-30T00:01:37.999Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 51 done
2024-01-30T00:01:38.030Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-01-30T00:01:38.040Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-01-30T00:01:38.093Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 59 done
2024-01-30T00:01:38.365Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 39 done
2024-01-30T00:01:38.450Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 43 done
2024-01-30T00:01:38.478Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-01-30T00:01:38.546Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 37 done
2024-01-30T00:01:39.046Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 55 done
2024-01-30T00:01:39.229Z	INFO	fil-consensus	filcns/upgrades.go:2418	2950188 jobs created, 2950155 done, 33 pending after 5m10.02743011s (9516/s)
2024-01-30T00:01:39.354Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-01-30T00:01:39.482Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-01-30T00:01:39.497Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 57 done
2024-01-30T00:01:39.554Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 47 done
2024-01-30T00:01:39.859Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-01-30T00:01:40.394Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-01-30T00:01:40.513Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 44 done
2024-01-30T00:01:40.634Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 48 done
2024-01-30T00:01:40.720Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-01-30T00:01:40.770Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-01-30T00:01:40.965Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-01-30T00:01:40.975Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-01-30T00:01:41.137Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 50 done
2024-01-30T00:01:41.283Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-01-30T00:01:41.372Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-01-30T00:01:41.535Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-01-30T00:01:42.150Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 38 done
2024-01-30T00:01:42.343Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-01-30T00:01:42.432Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 60 done
2024-01-30T00:01:43.016Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-01-30T00:01:43.569Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-01-30T00:01:43.938Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-01-30T00:01:44.080Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 46 done
2024-01-30T00:01:44.390Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 58 done
2024-01-30T00:01:44.585Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-01-30T00:01:45.571Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-01-30T00:01:45.614Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 54 done
2024-01-30T00:01:46.158Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 42 done
2024-01-30T00:01:48.781Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 49 done
2024-01-30T00:01:49.229Z	INFO	fil-consensus	filcns/upgrades.go:2418	2950188 jobs created, 2950184 done, 4 pending after 5m20.027711589s (9219/s)
2024-01-30T00:01:49.510Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-01-30T00:01:50.013Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-01-30T00:01:54.168Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-01-30T00:01:56.847Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 40 done
2024-01-30T00:01:56.847Z	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 5m27.646181584s
2024-01-30T00:01:56.847Z	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2950187 results to state tree after 5m27.646222664s
2024-01-30T00:01:56.848Z	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-01-30T00:01:56.848Z	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-01-30T00:04:19.769Z	INFO	fil-consensus	filcns/upgrades.go:2418	All 2950188 done after 7m50.567519263s (6269/s)
migration height  3607608
old cid  bafy2bzacearxqnq6ppd6w2uqri4un3eol3u3c2qvkmdfcmjokl4roi3yx37js
new cid  bafy2bzaceanctncjt6bmbsfvetzewnnihtpbg5r62bhnoewhtz55dsghyteoy
completed round actual (without cache), took  8m1.817901761s
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-01-30T00:04:31.324Z	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-01-30T00:04:31.324Z	INFO	fil-consensus	filcns/upgrades.go:2418	Started 32 workers
2024-01-30T00:04:31.325Z	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-01-30T00:09:31.325Z	INFO	fil-consensus	filcns/upgrades.go:2418	2742189 jobs created, 2742157 done, 32 pending after 5m0.000387719s (9141/s)
2024-01-30T00:09:53.539Z	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2949864 migration jobs after 5m22.214289441s
2024-01-30T00:09:53.539Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-01-30T00:09:53.539Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-01-30T00:09:53.539Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-01-30T00:09:53.539Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-01-30T00:09:53.540Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-01-30T00:09:53.573Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-01-30T00:09:53.580Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-01-30T00:09:53.724Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-01-30T00:09:53.755Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-01-30T00:09:53.765Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-01-30T00:09:53.885Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-01-30T00:09:54.035Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-01-30T00:09:54.216Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-01-30T00:09:54.234Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-01-30T00:09:54.303Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-01-30T00:09:54.336Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-01-30T00:09:54.421Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-01-30T00:09:54.522Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-01-30T00:09:55.013Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-01-30T00:09:55.222Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-01-30T00:09:55.240Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-01-30T00:09:55.319Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-01-30T00:09:55.567Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-01-30T00:09:55.574Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-01-30T00:09:56.333Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-01-30T00:09:56.507Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-01-30T00:09:56.584Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-01-30T00:09:56.663Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-01-30T00:09:57.233Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-01-30T00:09:57.331Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-01-30T00:09:59.844Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-01-30T00:10:04.314Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-01-30T00:10:04.314Z	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 5m32.989859323s
2024-01-30T00:10:04.314Z	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2949863 results to state tree after 5m32.989928473s
2024-01-30T00:10:04.314Z	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-01-30T00:10:04.314Z	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-01-30T00:11:57.543Z	INFO	fil-consensus	filcns/upgrades.go:2418	All 2949864 done after 7m26.218352124s (6611/s)
completed premigration, took  7m30.315611674s
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-01-30T00:12:01.285Z	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-01-30T00:12:01.285Z	INFO	fil-consensus	filcns/upgrades.go:2418	Started 61 workers
2024-01-30T00:12:01.285Z	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-01-30T00:12:11.285Z	INFO	fil-consensus	filcns/upgrades.go:2418	807217 jobs created, 807215 done, 2 pending after 10.000172992s (80720/s)
2024-01-30T00:12:21.285Z	INFO	fil-consensus	filcns/upgrades.go:2418	1825160 jobs created, 1825158 done, 2 pending after 20.000286142s (91257/s)
2024-01-30T00:12:31.285Z	INFO	fil-consensus	filcns/upgrades.go:2418	2819324 jobs created, 2819324 done, 0 pending after 30.000438876s (93976/s)
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2950188 migration jobs after 31.190021524s
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 57 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 39 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 38 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 55 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 45 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 47 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 60 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 41 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 59 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 51 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 46 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 44 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 50 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 53 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 43 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 40 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 49 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 58 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 52 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 37 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 56 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 42 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 48 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 54 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-01-30T00:12:32.475Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-01-30T00:12:32.476Z	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2950187 results to state tree after 31.191495193s
2024-01-30T00:12:32.476Z	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 31.191491413s
2024-01-30T00:12:32.476Z	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-01-30T00:12:32.476Z	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
missing deal?? weird market cron? 9568480
missing deal?? weird market cron? 9568481
missing deal?? weird market cron? 9568482
missing deal?? weird market cron? 9568483
missing deal?? weird market cron? 9568484
missing deal?? weird market cron? 9568485
missing deal?? weird market cron? 9568486
missing deal?? weird market cron? 9568487
missing deal?? weird market cron? 10087550
missing deal?? weird market cron? 10087551
missing deal?? weird market cron? 10087552
missing deal?? weird market cron? 10259955
missing deal?? weird market cron? 10259956
missing deal?? weird market cron? 10259957
missing deal?? weird market cron? 10259958
missing deal?? weird market cron? 11037073
missing deal?? weird market cron? 11037074
missing deal?? weird market cron? 11037075
missing deal?? weird market cron? 11037076
missing deal?? weird market cron? 11037077
missing deal?? weird market cron? 11296809
missing deal?? weird market cron? 11296814
missing deal?? weird market cron? 11383063
missing deal?? weird market cron? 11383064
missing deal?? weird market cron? 11383065
missing deal?? weird market cron? 11383066
missing deal?? weird market cron? 11383067
missing deal?? weird market cron? 11383068
missing deal?? weird market cron? 11383069
missing deal?? weird market cron? 11383070
missing deal?? weird market cron? 11901969
missing deal?? weird market cron? 11901970
missing deal?? weird market cron? 11901971
missing deal?? weird market cron? 11901972
missing deal?? weird market cron? 11901973
missing deal?? weird market cron? 11901974
missing deal?? weird market cron? 11901975
missing deal?? weird market cron? 11901976
missing deal?? weird market cron? 12074487
missing deal?? weird market cron? 12074488
missing deal?? weird market cron? 12160598
missing deal?? weird market cron? 12160599
missing deal?? weird market cron? 12160600
missing deal?? weird market cron? 12160601
missing deal?? weird market cron? 12160602
missing deal?? weird market cron? 12160603
missing deal?? weird market cron? 12333230
missing deal?? weird market cron? 12419947
missing deal?? weird market cron? 12419948
missing deal?? weird market cron? 12419949
missing deal?? weird market cron? 12419950
missing deal?? weird market cron? 12419951
missing deal?? weird market cron? 12419952
missing deal?? weird market cron? 12419953
missing deal?? weird market cron? 12419954
missing deal?? weird market cron? 12678961
missing deal?? weird market cron? 14061147
missing deal?? weird market cron? 14061149
missing deal?? weird market cron? 14148280
missing deal?? weird market cron? 14406852
missing deal?? weird market cron? 14406853
missing deal?? weird market cron? 14406854
missing deal?? weird market cron? 14406855
missing deal?? weird market cron? 14579993
missing deal?? weird market cron? 14579994
missing deal?? weird market cron? 14579995
missing deal?? weird market cron? 14579996
missing deal?? weird market cron? 14579997
missing deal?? weird market cron? 14579998
missing deal?? weird market cron? 14579999
missing deal?? weird market cron? 14925332
missing deal?? weird market cron? 15875724
missing deal?? weird market cron? 16048563
missing deal?? weird market cron? 16048566
missing deal?? weird market cron? 16566806
missing deal?? weird market cron? 16826657
missing deal?? weird market cron? 17517939
missing deal?? weird market cron? 17517940
missing deal?? weird market cron? 17517941
missing deal?? weird market cron? 18208371
missing deal?? weird market cron? 18208372
missing deal?? weird market cron? 18208373
missing deal?? weird market cron? 18208374
missing deal?? weird market cron? 18468350
missing deal?? weird market cron? 20023180
missing deal?? weird market cron? 20023181
missing deal?? weird market cron? 20023182
missing deal?? weird market cron? 20023183
missing deal?? weird market cron? 20023184
missing deal?? weird market cron? 20023185
missing deal?? weird market cron? 20023186
missing deal?? weird market cron? 20023187
missing deal?? weird market cron? 31686734
missing deal?? weird market cron? 31686738
missing deal?? weird market cron? 42055473
missing deal?? weird market cron? 42055485
missing deal?? weird market cron? 42055558
missing deal?? weird market cron? 42055559
missing deal?? weird market cron? 42055560
missing deal?? weird market cron? 42055561
missing deal?? weird market cron? 42055562
missing deal?? weird market cron? 42055563
missing deal?? weird market cron? 42055564
missing deal?? weird market cron? 42055565
missing deal?? weird market cron? 42055566
missing deal?? weird market cron? 42055567
missing deal?? weird market cron? 42055568
missing deal?? weird market cron? 42055569
missing deal?? weird market cron? 42055570
missing deal?? weird market cron? 42055571
missing deal?? weird market cron? 42055572
missing deal?? weird market cron? 42055573
missing deal?? weird market cron? 42055574
missing deal?? weird market cron? 42055575
missing deal?? weird market cron? 42055576
missing deal?? weird market cron? 42055577
missing deal?? weird market cron? 42055578
missing deal?? weird market cron? 42055579
missing deal?? weird market cron? 42055580
missing deal?? weird market cron? 42055581
missing deal?? weird market cron? 42055582
missing deal?? weird market cron? 42400270
missing deal?? weird market cron? 44042614
missing deal?? weird market cron? 44042627
missing deal?? weird market cron? 47843555
missing deal?? weird market cron? 47843556
missing deal?? weird market cron? 47843557
missing deal?? weird market cron? 47843558
missing deal?? weird market cron? 47843559
missing deal?? weird market cron? 47843560
missing deal?? weird market cron? 47843561
missing deal?? weird market cron? 48621481
missing deal?? weird market cron? 48621482
missing deal?? weird market cron? 48621483
missing deal?? weird market cron? 48621484
missing deal?? weird market cron? 48621485
missing deal?? weird market cron? 48621486
missing deal?? weird market cron? 48621487
missing deal?? weird market cron? 48621488
missing deal?? weird market cron? 52682022
missing deal?? weird market cron? 67110758
missing deal?? weird market cron? 67110780
missing deal?? weird market cron? 67110895
missing deal?? weird market cron? 67110897
missing deal?? weird market cron? 67110906
missing deal?? weird market cron? 67110927
missing deal?? weird market cron? 67110937
missing deal?? weird market cron? 67110982
missing deal?? weird market cron? 67111604
missing deal?? weird market cron? 67111607
2024-01-30T00:12:36.342Z	INFO	fil-consensus	filcns/upgrades.go:2418	All 2950188 done after 35.057035301s (84154/s)
completed round actual (with cache), took  38.627615935s

@magik6k magik6k force-pushed the feat/nv22-migration-bench branch from 141bfd5 to 3798e15 Compare February 6, 2024 11:11
@rjan90
Copy link
Contributor

rjan90 commented Feb 8, 2024

Works with the latest commit (fb19028):

./lotus-shed migrate-state --repo=/mnt/lotuschain/.lotus 22 bafy2bzaceag7ixqzbemohrmb2qbj2b5em5erotldfvn3ojanumi4lfnsl2wxi
REMINDER: If you are running this, you likely want to ALSO run the continuity testing tool!
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-02-08T16:47:04.246+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Started 37 workers
2024-02-08T16:47:04.246+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-02-08T16:47:04.246+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-02-08T16:47:14.247+0100	INFO	fil-consensus	filcns/upgrades.go:2418	128209 jobs created, 127172 done, 1037 pending after 10.000408793s (12717/s)
2024-02-08T16:47:24.247+0100	INFO	fil-consensus	filcns/upgrades.go:2418	220607 jobs created, 219570 done, 1037 pending after 20.000858921s (10978/s)
2024-02-08T16:47:34.247+0100	INFO	fil-consensus	filcns/upgrades.go:2418	323954 jobs created, 323111 done, 843 pending after 30.00106194s (10770/s)
2024-02-08T16:47:44.247+0100	INFO	fil-consensus	filcns/upgrades.go:2418	432778 jobs created, 431741 done, 1037 pending after 40.001339816s (10793/s)
2024-02-08T16:47:54.248+0100	INFO	fil-consensus	filcns/upgrades.go:2418	564716 jobs created, 564085 done, 631 pending after 50.001818123s (11281/s)
2024-02-08T16:48:04.249+0100	INFO	fil-consensus	filcns/upgrades.go:2418	583293 jobs created, 582256 done, 1037 pending after 1m0.002963602s (9704/s)
2024-02-08T16:48:14.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	634916 jobs created, 633879 done, 1037 pending after 1m10.015584451s (9053/s)
2024-02-08T16:48:24.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	749169 jobs created, 748149 done, 1020 pending after 1m20.015760745s (9350/s)
2024-02-08T16:48:34.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	820218 jobs created, 819181 done, 1037 pending after 1m30.016288864s (9100/s)
2024-02-08T16:48:44.263+0100	INFO	fil-consensus	filcns/upgrades.go:2418	927333 jobs created, 926296 done, 1037 pending after 1m40.016687336s (9261/s)
2024-02-08T16:48:54.263+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1032478 jobs created, 1031441 done, 1037 pending after 1m50.017034503s (9375/s)
2024-02-08T16:49:04.263+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1115508 jobs created, 1114471 done, 1037 pending after 2m0.017322035s (9286/s)
2024-02-08T16:49:14.264+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1161675 jobs created, 1160638 done, 1037 pending after 2m10.017534737s (8927/s)
2024-02-08T16:49:24.264+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1194281 jobs created, 1193244 done, 1037 pending after 2m20.017772011s (8522/s)
2024-02-08T16:49:34.264+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1249547 jobs created, 1248605 done, 942 pending after 2m30.018050184s (8323/s)
2024-02-08T16:49:44.264+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1368217 jobs created, 1367180 done, 1037 pending after 2m40.018220376s (8544/s)
2024-02-08T16:49:54.265+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1483622 jobs created, 1482642 done, 980 pending after 2m50.018485559s (8720/s)
2024-02-08T16:50:04.267+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1593552 jobs created, 1592520 done, 1032 pending after 3m0.021039498s (8846/s)
2024-02-08T16:50:14.267+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1696461 jobs created, 1695424 done, 1037 pending after 3m10.021252844s (8922/s)
2024-02-08T16:50:24.268+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1805292 jobs created, 1805079 done, 213 pending after 3m20.021743767s (9024/s)
2024-02-08T16:50:34.270+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1899089 jobs created, 1898059 done, 1030 pending after 3m30.023420853s (9037/s)
2024-02-08T16:50:44.271+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1998534 jobs created, 1997497 done, 1037 pending after 3m40.025068774s (9078/s)
2024-02-08T16:50:54.272+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2078253 jobs created, 2077216 done, 1037 pending after 3m50.025607416s (9030/s)
2024-02-08T16:51:04.273+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2172861 jobs created, 2171824 done, 1037 pending after 4m0.026427395s (9048/s)
2024-02-08T16:51:14.273+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2237957 jobs created, 2236920 done, 1037 pending after 4m10.02679548s (8947/s)
2024-02-08T16:51:24.283+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2300220 jobs created, 2299263 done, 957 pending after 4m20.036593668s (8842/s)
2024-02-08T16:51:34.283+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2352272 jobs created, 2351235 done, 1037 pending after 4m30.036727424s (8707/s)
2024-02-08T16:51:44.283+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2362235 jobs created, 2361198 done, 1037 pending after 4m40.037079051s (8432/s)
2024-02-08T16:51:54.283+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2378301 jobs created, 2377264 done, 1037 pending after 4m50.037333078s (8196/s)
2024-02-08T16:52:04.284+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2410643 jobs created, 2409606 done, 1037 pending after 5m0.037607128s (8031/s)
2024-02-08T16:52:14.286+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2481629 jobs created, 2480613 done, 1016 pending after 5m10.040282496s (8001/s)
2024-02-08T16:52:24.287+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2573703 jobs created, 2572666 done, 1037 pending after 5m20.040563265s (8039/s)
2024-02-08T16:52:34.289+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2680101 jobs created, 2679263 done, 838 pending after 5m30.043259192s (8118/s)
2024-02-08T16:52:44.290+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2774225 jobs created, 2773245 done, 980 pending after 5m40.04383482s (8156/s)
2024-02-08T16:52:54.290+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2857177 jobs created, 2856140 done, 1037 pending after 5m50.044015996s (8159/s)
2024-02-08T16:53:03.985+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2957131 migration jobs after 5m59.731551024s
2024-02-08T16:53:04.013+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-02-08T16:53:04.013+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-02-08T16:53:04.013+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-02-08T16:53:04.073+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-02-08T16:53:04.141+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-02-08T16:53:04.197+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-02-08T16:53:04.237+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-02-08T16:53:04.255+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-02-08T16:53:04.290+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2957131 jobs created, 2957102 done, 29 pending after 6m0.044180973s (8213/s)
2024-02-08T16:53:04.291+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-02-08T16:53:04.306+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-02-08T16:53:04.440+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-02-08T16:53:04.536+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-02-08T16:53:04.560+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-02-08T16:53:04.657+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-02-08T16:53:04.792+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-02-08T16:53:04.893+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-02-08T16:53:04.904+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-02-08T16:53:04.974+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-02-08T16:53:05.006+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-02-08T16:53:05.553+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-02-08T16:53:05.742+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-02-08T16:53:05.780+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-02-08T16:53:06.006+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-02-08T16:53:06.084+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-02-08T16:53:06.312+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-02-08T16:53:06.686+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-02-08T16:53:06.766+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-02-08T16:53:07.794+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-02-08T16:53:08.019+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-02-08T16:53:08.026+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-02-08T16:53:08.300+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-02-08T16:53:08.651+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-02-08T16:53:08.773+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-02-08T16:53:09.059+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-02-08T16:53:09.266+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-02-08T16:53:13.910+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-02-08T16:53:14.290+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2957131 jobs created, 2957130 done, 1 pending after 6m10.044343703s (7991/s)
2024-02-08T16:53:21.179+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-02-08T16:53:21.187+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2957130 results to state tree after 6m16.933126977s
2024-02-08T16:53:21.187+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 6m16.933112695s
2024-02-08T16:53:21.187+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-02-08T16:53:21.190+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-02-08T17:00:33.384+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All 2957131 done after 13m29.130902635s (3655/s)
migration height  3636691
old cid  bafy2bzaceajbehzjnhngnbfl6tkafff7afcs2pb37lyzq7u6ztjqv4w5co2gq
new cid  bafy2bzacedxcscwbheft3f7tgoiqgcgm2fawfpyprnqges3bmic66h7eddayu
completed round actual (without cache), took  13m36.245578801s
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-02-08T17:00:40.798+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-02-08T17:00:40.798+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Started 20 workers
2024-02-08T17:00:40.799+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-02-08T17:05:40.811+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2293771 jobs created, 2293751 done, 20 pending after 5m0.000441787s (7646/s)
2024-02-08T17:07:29.446+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2956864 migration jobs after 6m48.640615202s
2024-02-08T17:07:29.446+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-02-08T17:07:29.446+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-02-08T17:07:29.446+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-02-08T17:07:29.455+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-02-08T17:07:29.911+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-02-08T17:07:29.940+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-02-08T17:07:30.111+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-02-08T17:07:30.210+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-02-08T17:07:30.372+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-02-08T17:07:30.798+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-02-08T17:07:30.911+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-02-08T17:07:31.390+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-02-08T17:07:31.689+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-02-08T17:07:31.860+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-02-08T17:07:32.202+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-02-08T17:07:32.706+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-02-08T17:07:33.080+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-02-08T17:07:33.352+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-02-08T17:07:33.921+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-02-08T17:07:41.761+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-02-08T17:07:41.761+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2956863 results to state tree after 7m0.962712502s
2024-02-08T17:07:41.768+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 7m0.962610276s
2024-02-08T17:07:41.768+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-02-08T17:07:41.769+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-02-08T17:14:45.021+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All 2956864 done after 14m4.222512374s (3502/s)
completed premigration, took  14m9.288776119s
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-02-08T17:14:49.707+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-02-08T17:14:49.707+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Started 37 workers
2024-02-08T17:14:49.709+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-02-08T17:14:59.708+0100	INFO	fil-consensus	filcns/upgrades.go:2418	377093 jobs created, 377093 done, 0 pending after 10.001350077s (37704/s)
2024-02-08T17:15:09.708+0100	INFO	fil-consensus	filcns/upgrades.go:2418	916881 jobs created, 916879 done, 2 pending after 20.001650587s (45840/s)
2024-02-08T17:15:19.709+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1465317 jobs created, 1465316 done, 1 pending after 30.001824568s (48841/s)
2024-02-08T17:15:29.709+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2162708 jobs created, 2162703 done, 5 pending after 40.002050503s (54065/s)
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2957131 migration jobs after 49.458593698s
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-02-08T17:15:39.165+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-02-08T17:15:39.166+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-02-08T17:15:39.167+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2957130 results to state tree after 49.459863803s
2024-02-08T17:15:39.167+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 49.459854565s
2024-02-08T17:15:39.167+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-02-08T17:15:39.167+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-02-08T17:15:47.936+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All 2957131 done after 58.229132459s (50784/s)
completed round actual (with cache), took  1m2.638565234s

@TippyFlitsUK
Copy link
Contributor

./lotus-shed migrate-state --repo=/filecoin-sealing/.lotus-daemon 22 bafy2bzaceag7ixqzbemohrmb2qbj2b5em5erotldfvn3ojanumi4lfnsl2wxi
REMINDER: If you are running this, you likely want to ALSO run the continuity testing tool!
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-02-08T16:46:19.500Z	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-02-08T16:46:19.501Z	INFO	fil-consensus	filcns/upgrades.go:2418	Started 61 workers
2024-02-08T16:46:19.501Z	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-02-08T16:46:29.501Z	INFO	fil-consensus	filcns/upgrades.go:2418	160570 jobs created, 160009 done, 561 pending after 10.000924112s (15999/s)
2024-02-08T16:46:39.501Z	INFO	fil-consensus	filcns/upgrades.go:2418	267826 jobs created, 266765 done, 1061 pending after 20.00118966s (13337/s)
2024-02-08T16:46:49.502Z	INFO	fil-consensus	filcns/upgrades.go:2418	380754 jobs created, 379720 done, 1034 pending after 30.001772709s (12657/s)
2024-02-08T16:46:59.503Z	INFO	fil-consensus	filcns/upgrades.go:2418	505270 jobs created, 504464 done, 806 pending after 40.002418749s (12611/s)
2024-02-08T16:47:09.503Z	INFO	fil-consensus	filcns/upgrades.go:2418	602296 jobs created, 601239 done, 1057 pending after 50.002687356s (12024/s)
2024-02-08T16:47:19.504Z	INFO	fil-consensus	filcns/upgrades.go:2418	679466 jobs created, 678405 done, 1061 pending after 1m0.003025034s (11306/s)
2024-02-08T16:47:29.505Z	INFO	fil-consensus	filcns/upgrades.go:2418	806034 jobs created, 805089 done, 945 pending after 1m10.004736349s (11500/s)
2024-02-08T16:47:39.505Z	INFO	fil-consensus	filcns/upgrades.go:2418	913957 jobs created, 912896 done, 1061 pending after 1m20.004919235s (11410/s)
2024-02-08T16:47:49.506Z	INFO	fil-consensus	filcns/upgrades.go:2418	1003150 jobs created, 1002354 done, 796 pending after 1m30.005721585s (11137/s)
2024-02-08T16:47:59.507Z	INFO	fil-consensus	filcns/upgrades.go:2418	1094991 jobs created, 1093930 done, 1061 pending after 1m40.006973067s (10939/s)
2024-02-08T16:48:09.507Z	INFO	fil-consensus	filcns/upgrades.go:2418	1169761 jobs created, 1168700 done, 1061 pending after 1m50.007254983s (10624/s)
2024-02-08T16:48:19.508Z	INFO	fil-consensus	filcns/upgrades.go:2418	1211970 jobs created, 1210909 done, 1061 pending after 2m0.007554728s (10090/s)
2024-02-08T16:48:29.508Z	INFO	fil-consensus	filcns/upgrades.go:2418	1266966 jobs created, 1265905 done, 1061 pending after 2m10.007908454s (9737/s)
2024-02-08T16:48:39.508Z	INFO	fil-consensus	filcns/upgrades.go:2418	1363703 jobs created, 1362928 done, 775 pending after 2m20.008230469s (9735/s)
2024-02-08T16:48:49.509Z	INFO	fil-consensus	filcns/upgrades.go:2418	1507440 jobs created, 1506382 done, 1058 pending after 2m30.009184138s (10042/s)
2024-02-08T16:48:59.510Z	INFO	fil-consensus	filcns/upgrades.go:2418	1639231 jobs created, 1638188 done, 1043 pending after 2m40.009450783s (10238/s)
2024-02-08T16:49:09.510Z	INFO	fil-consensus	filcns/upgrades.go:2418	1750934 jobs created, 1749887 done, 1047 pending after 2m50.01011939s (10293/s)
2024-02-08T16:49:19.510Z	INFO	fil-consensus	filcns/upgrades.go:2418	1876337 jobs created, 1875482 done, 855 pending after 3m0.010298113s (10419/s)
2024-02-08T16:49:29.513Z	INFO	fil-consensus	filcns/upgrades.go:2418	1963476 jobs created, 1962415 done, 1061 pending after 3m10.012971682s (10328/s)
2024-02-08T16:49:39.514Z	INFO	fil-consensus	filcns/upgrades.go:2418	2072570 jobs created, 2071509 done, 1061 pending after 3m20.013540527s (10357/s)
2024-02-08T16:49:49.515Z	INFO	fil-consensus	filcns/upgrades.go:2418	2157048 jobs created, 2155987 done, 1061 pending after 3m30.014776917s (10266/s)
2024-02-08T16:49:59.516Z	INFO	fil-consensus	filcns/upgrades.go:2418	2259295 jobs created, 2258234 done, 1061 pending after 3m40.015398962s (10264/s)
2024-02-08T16:50:09.516Z	INFO	fil-consensus	filcns/upgrades.go:2418	2349679 jobs created, 2348618 done, 1061 pending after 3m50.015708925s (10211/s)
2024-02-08T16:50:19.516Z	INFO	fil-consensus	filcns/upgrades.go:2418	2397271 jobs created, 2396210 done, 1061 pending after 4m0.016005828s (9984/s)
2024-02-08T16:50:29.517Z	INFO	fil-consensus	filcns/upgrades.go:2418	2432208 jobs created, 2431147 done, 1061 pending after 4m10.01651758s (9724/s)
2024-02-08T16:50:39.517Z	INFO	fil-consensus	filcns/upgrades.go:2418	2489584 jobs created, 2488649 done, 935 pending after 4m20.016858335s (9571/s)
2024-02-08T16:50:49.518Z	INFO	fil-consensus	filcns/upgrades.go:2418	2553758 jobs created, 2552697 done, 1061 pending after 4m30.017976544s (9454/s)
2024-02-08T16:50:59.518Z	INFO	fil-consensus	filcns/upgrades.go:2418	2666237 jobs created, 2665176 done, 1061 pending after 4m40.018174816s (9518/s)
2024-02-08T16:51:09.519Z	INFO	fil-consensus	filcns/upgrades.go:2418	2784587 jobs created, 2783527 done, 1060 pending after 4m50.018452116s (9598/s)
2024-02-08T16:51:19.520Z	INFO	fil-consensus	filcns/upgrades.go:2418	2895613 jobs created, 2894624 done, 989 pending after 5m0.019564321s (9648/s)
2024-02-08T16:51:25.009Z	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2957131 migration jobs after 5m5.508813198s
2024-02-08T16:51:25.042Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-02-08T16:51:25.042Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 45 done
2024-02-08T16:51:25.042Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-02-08T16:51:25.042Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 50 done
2024-02-08T16:51:25.042Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-02-08T16:51:25.043Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 52 done
2024-02-08T16:51:25.066Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 60 done
2024-02-08T16:51:25.099Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-02-08T16:51:25.100Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 46 done
2024-02-08T16:51:25.168Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-02-08T16:51:25.169Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-02-08T16:51:25.361Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 37 done
2024-02-08T16:51:25.371Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 57 done
2024-02-08T16:51:25.492Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 48 done
2024-02-08T16:51:25.497Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-02-08T16:51:25.555Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-02-08T16:51:25.577Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 39 done
2024-02-08T16:51:25.578Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-02-08T16:51:25.578Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-02-08T16:51:25.735Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-02-08T16:51:25.741Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-02-08T16:51:25.743Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 49 done
2024-02-08T16:51:25.772Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-02-08T16:51:26.041Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-02-08T16:51:26.159Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-02-08T16:51:26.217Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-02-08T16:51:26.240Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-02-08T16:51:26.443Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-02-08T16:51:26.454Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-02-08T16:51:26.469Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 59 done
2024-02-08T16:51:26.562Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-02-08T16:51:26.780Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-02-08T16:51:27.162Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-02-08T16:51:27.377Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 58 done
2024-02-08T16:51:27.382Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-02-08T16:51:27.397Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-02-08T16:51:27.447Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-02-08T16:51:27.742Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-02-08T16:51:27.837Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 47 done
2024-02-08T16:51:27.962Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 40 done
2024-02-08T16:51:28.064Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 53 done
2024-02-08T16:51:28.193Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-02-08T16:51:28.259Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-02-08T16:51:28.336Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 54 done
2024-02-08T16:51:29.074Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 51 done
2024-02-08T16:51:29.143Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 42 done
2024-02-08T16:51:29.520Z	INFO	fil-consensus	filcns/upgrades.go:2418	2957131 jobs created, 2957116 done, 15 pending after 5m10.019751977s (9538/s)
2024-02-08T16:51:29.803Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-02-08T16:51:30.316Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-02-08T16:51:30.712Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-02-08T16:51:30.712Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 55 done
2024-02-08T16:51:30.913Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-02-08T16:51:31.472Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-02-08T16:51:31.710Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 38 done
2024-02-08T16:51:32.125Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-02-08T16:51:32.371Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 43 done
2024-02-08T16:51:32.839Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 56 done
2024-02-08T16:51:36.245Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-02-08T16:51:36.549Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-02-08T16:51:36.724Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 41 done
2024-02-08T16:51:39.520Z	INFO	fil-consensus	filcns/upgrades.go:2418	2957131 jobs created, 2957129 done, 2 pending after 5m20.019941443s (9240/s)
2024-02-08T16:51:39.647Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 44 done
2024-02-08T16:51:42.154Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-02-08T16:51:42.154Z	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 5m22.654238614s
2024-02-08T16:51:42.154Z	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2957130 results to state tree after 5m22.654257514s
2024-02-08T16:51:42.155Z	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-02-08T16:51:42.155Z	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-02-08T16:57:17.421Z	INFO	fil-consensus	filcns/upgrades.go:2418	All 2957131 done after 10m57.920391732s (4495/s)
migration height  3636691
old cid  bafy2bzaceajbehzjnhngnbfl6tkafff7afcs2pb37lyzq7u6ztjqv4w5co2gq
new cid  bafy2bzacedxcscwbheft3f7tgoiqgcgm2fawfpyprnqges3bmic66h7eddayu
completed round actual (without cache), took  11m5.026856379s
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-02-08T16:57:24.663Z	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-02-08T16:57:24.663Z	INFO	fil-consensus	filcns/upgrades.go:2418	Started 32 workers
2024-02-08T16:57:24.663Z	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-02-08T17:02:24.663Z	INFO	fil-consensus	filcns/upgrades.go:2418	2677867 jobs created, 2677837 done, 30 pending after 5m0.000202525s (8926/s)
2024-02-08T17:02:55.326Z	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2956864 migration jobs after 5m30.663032459s
2024-02-08T17:02:55.326Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-02-08T17:02:55.326Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-02-08T17:02:55.326Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-02-08T17:02:55.326Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-02-08T17:02:55.326Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-02-08T17:02:55.326Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-02-08T17:02:55.506Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-02-08T17:02:55.615Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-02-08T17:02:55.631Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-02-08T17:02:55.671Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-02-08T17:02:55.673Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-02-08T17:02:55.968Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-02-08T17:02:56.089Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-02-08T17:02:56.104Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-02-08T17:02:56.158Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-02-08T17:02:56.263Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-02-08T17:02:56.278Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-02-08T17:02:56.419Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-02-08T17:02:56.549Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-02-08T17:02:56.863Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-02-08T17:02:56.961Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-02-08T17:02:57.034Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-02-08T17:02:57.520Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-02-08T17:02:57.722Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-02-08T17:02:58.627Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-02-08T17:02:58.892Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-02-08T17:02:58.922Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-02-08T17:02:59.061Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-02-08T17:02:59.537Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-02-08T17:02:59.739Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-02-08T17:03:02.457Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-02-08T17:03:05.618Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-02-08T17:03:05.618Z	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 5m40.95562988s
2024-02-08T17:03:05.618Z	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2956863 results to state tree after 5m40.955679581s
2024-02-08T17:03:05.618Z	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-02-08T17:03:05.618Z	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-02-08T17:08:25.000Z	INFO	fil-consensus	filcns/upgrades.go:2418	All 2956864 done after 11m0.337512651s (4478/s)
completed premigration, took  11m3.648146132s
manifest cid: bafy2bzacecpuup2nsfmmrrkm4fb4bvm4gcmhvoslhzmnz36w5teecs2mrvhjg
2024-02-08T17:08:28.117Z	INFO	fil-consensus	filcns/upgrades.go:2418	Started 61 workers
2024-02-08T17:08:28.117Z	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-02-08T17:08:28.117Z	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-02-08T17:08:38.117Z	INFO	fil-consensus	filcns/upgrades.go:2418	729580 jobs created, 729565 done, 15 pending after 10.000189354s (72955/s)
2024-02-08T17:08:48.117Z	INFO	fil-consensus	filcns/upgrades.go:2418	1835673 jobs created, 1835672 done, 1 pending after 20.000291337s (91782/s)
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2957131 migration jobs after 29.469014634s
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 60 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 41 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 44 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 38 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 48 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 40 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 57 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 42 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 54 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 51 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 45 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 53 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 56 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 47 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 46 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 50 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 37 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 52 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 59 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 58 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 55 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 49 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 39 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 43 done
2024-02-08T17:08:57.586Z	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-02-08T17:08:57.587Z	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 29.469998221s
2024-02-08T17:08:57.587Z	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2957130 results to state tree after 29.470052381s
2024-02-08T17:08:57.587Z	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-02-08T17:08:57.587Z	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-02-08T17:09:04.484Z	INFO	fil-consensus	filcns/upgrades.go:2418	All 2957131 done after 36.367625104s (81312/s)
completed round actual (with cache), took  42.854179418s

@TippyFlitsUK
Copy link
Contributor

TippyFlitsUK commented Feb 9, 2024

Calibration test:
calib_logs.txt

@TippyFlitsUK
Copy link
Contributor

Mainnet logs:
mainnet_logs.txt

@TippyFlitsUK
Copy link
Contributor

Mainnet logs with invariants:
nv22-invariants..log

@Reiers
Copy link

Reiers commented Feb 12, 2024

./lotus-shed migrate-state --repo=/mnt/Lotus_Friday/lotus_test_node 22 bafy2bzacebcf7glgi5r5ng776rcsagn7rderyuwlzo4iv3edcjbxdu6drrdnw
REMINDER: If you are running this, you likely want to ALSO run the continuity testing tool!
manifest cid: bafy2bzacea6f5icdp6t6fs5sexjxmo3q5d2qu4g4ghq6s5eaob6svnmhvltmw
2024-02-12T15:55:30.054+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-02-12T15:55:30.054+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Started 125 workers
2024-02-12T15:55:30.055+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-02-12T15:55:40.055+0100	INFO	fil-consensus	filcns/upgrades.go:2418	150500 jobs created, 150439 done, 61 pending after 10.000918961s (15043/s)
2024-02-12T15:55:50.056+0100	INFO	fil-consensus	filcns/upgrades.go:2418	279501 jobs created, 279446 done, 55 pending after 20.002051155s (13971/s)
2024-02-12T15:56:00.056+0100	INFO	fil-consensus	filcns/upgrades.go:2418	315485 jobs created, 315438 done, 47 pending after 30.002196714s (10514/s)
2024-02-12T15:56:10.056+0100	INFO	fil-consensus	filcns/upgrades.go:2418	362175 jobs created, 362123 done, 52 pending after 40.002464635s (9053/s)
2024-02-12T15:56:20.057+0100	INFO	fil-consensus	filcns/upgrades.go:2418	427684 jobs created, 427628 done, 56 pending after 50.002609904s (8552/s)
2024-02-12T15:56:30.057+0100	INFO	fil-consensus	filcns/upgrades.go:2418	479832 jobs created, 479774 done, 58 pending after 1m0.002758469s (7996/s)
2024-02-12T15:56:40.057+0100	INFO	fil-consensus	filcns/upgrades.go:2418	555245 jobs created, 555185 done, 60 pending after 1m10.003588114s (7931/s)
2024-02-12T15:56:50.058+0100	INFO	fil-consensus	filcns/upgrades.go:2418	614677 jobs created, 614626 done, 51 pending after 1m20.003729356s (7682/s)
2024-02-12T15:57:00.058+0100	INFO	fil-consensus	filcns/upgrades.go:2418	685431 jobs created, 685373 done, 58 pending after 1m30.004321331s (7615/s)
2024-02-12T15:57:10.058+0100	INFO	fil-consensus	filcns/upgrades.go:2418	743479 jobs created, 743425 done, 54 pending after 1m40.004462063s (7434/s)
2024-02-12T15:57:20.058+0100	INFO	fil-consensus	filcns/upgrades.go:2418	794227 jobs created, 794174 done, 53 pending after 1m50.004585051s (7219/s)
2024-02-12T15:57:30.059+0100	INFO	fil-consensus	filcns/upgrades.go:2418	905252 jobs created, 905169 done, 83 pending after 2m0.004796688s (7543/s)
2024-02-12T15:57:40.059+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1072275 jobs created, 1072175 done, 100 pending after 2m10.00501605s (8247/s)
2024-02-12T15:57:50.059+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1203844 jobs created, 1202719 done, 1125 pending after 2m20.005249148s (8591/s)
2024-02-12T15:58:00.059+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1250230 jobs created, 1250134 done, 96 pending after 2m30.005467659s (8334/s)
2024-02-12T15:58:10.059+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1303441 jobs created, 1303370 done, 71 pending after 2m40.005582564s (8146/s)
2024-02-12T15:58:20.060+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1436769 jobs created, 1436676 done, 93 pending after 2m50.005735803s (8451/s)
2024-02-12T15:58:30.060+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1631539 jobs created, 1631449 done, 90 pending after 3m0.006323834s (9063/s)
2024-02-12T15:58:40.060+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1832807 jobs created, 1832732 done, 75 pending after 3m10.006521427s (9646/s)
2024-02-12T15:58:50.061+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1995271 jobs created, 1995168 done, 103 pending after 3m20.006873031s (9975/s)
2024-02-12T15:59:00.061+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2152300 jobs created, 2152211 done, 89 pending after 3m30.007258671s (10248/s)
2024-02-12T15:59:10.062+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2275145 jobs created, 2274103 done, 1042 pending after 3m40.008206785s (10336/s)
2024-02-12T15:59:20.076+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2391912 jobs created, 2390832 done, 1080 pending after 3m50.022577119s (10394/s)
2024-02-12T15:59:30.077+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2521336 jobs created, 2520211 done, 1125 pending after 4m0.022817014s (10500/s)
2024-02-12T15:59:40.078+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2632385 jobs created, 2632296 done, 89 pending after 4m10.024227894s (10528/s)
2024-02-12T15:59:50.078+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2689766 jobs created, 2689709 done, 57 pending after 4m20.02435701s (10344/s)
2024-02-12T16:00:00.078+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2746770 jobs created, 2746706 done, 64 pending after 4m30.024558273s (10172/s)
2024-02-12T16:00:10.079+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2906119 jobs created, 2906040 done, 79 pending after 4m40.024696176s (10378/s)
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2948363 migration jobs after 4m44.340589541s
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 85 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 55 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 99 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 65 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 51 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 45 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 107 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 50 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 112 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 69 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 37 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 101 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 113 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 62 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 123 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 47 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 108 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 78 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 98 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 89 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 77 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 84 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 82 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 75 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 105 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 42 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 73 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 74 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 79 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 83 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 109 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 72 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 41 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 59 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 76 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 44 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 87 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 86 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 48 done
2024-02-12T16:00:14.395+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-02-12T16:00:14.410+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 39 done
2024-02-12T16:00:14.497+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 43 done
2024-02-12T16:00:14.527+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 110 done
2024-02-12T16:00:14.540+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 93 done
2024-02-12T16:00:14.604+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 91 done
2024-02-12T16:00:14.617+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-02-12T16:00:14.673+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 115 done
2024-02-12T16:00:14.713+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-02-12T16:00:14.823+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-02-12T16:00:14.859+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-02-12T16:00:14.925+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 58 done
2024-02-12T16:00:14.954+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 124 done
2024-02-12T16:00:15.223+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-02-12T16:00:15.251+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 67 done
2024-02-12T16:00:15.258+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 52 done
2024-02-12T16:00:15.402+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-02-12T16:00:15.569+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 46 done
2024-02-12T16:00:16.049+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-02-12T16:00:16.263+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-02-12T16:00:16.502+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-02-12T16:00:16.613+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 71 done
2024-02-12T16:00:16.721+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 88 done
2024-02-12T16:00:16.991+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 114 done
2024-02-12T16:00:17.062+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 111 done
2024-02-12T16:00:17.077+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 56 done
2024-02-12T16:00:17.336+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 54 done
2024-02-12T16:00:17.597+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 100 done
2024-02-12T16:00:17.987+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 90 done
2024-02-12T16:00:18.317+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 68 done
2024-02-12T16:00:18.464+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 117 done
2024-02-12T16:00:18.468+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-02-12T16:00:18.470+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 103 done
2024-02-12T16:00:18.959+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 38 done
2024-02-12T16:00:19.089+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 118 done
2024-02-12T16:00:19.109+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 49 done
2024-02-12T16:00:19.178+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-02-12T16:00:19.354+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 102 done
2024-02-12T16:00:19.981+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 95 done
2024-02-12T16:00:20.017+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 92 done
2024-02-12T16:00:20.024+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 61 done
2024-02-12T16:00:20.079+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2948363 jobs created, 2948328 done, 35 pending after 4m50.024960489s (10166/s)
2024-02-12T16:00:20.153+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 106 done
2024-02-12T16:00:20.218+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 80 done
2024-02-12T16:00:20.255+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-02-12T16:00:20.294+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 64 done
2024-02-12T16:00:20.420+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-02-12T16:00:20.447+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-02-12T16:00:20.515+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 53 done
2024-02-12T16:00:20.533+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 57 done
2024-02-12T16:00:20.596+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 60 done
2024-02-12T16:00:20.616+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-02-12T16:00:20.709+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 70 done
2024-02-12T16:00:20.798+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 81 done
2024-02-12T16:00:20.799+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 96 done
2024-02-12T16:00:20.956+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 40 done
2024-02-12T16:00:21.028+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-02-12T16:00:21.055+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-02-12T16:00:21.084+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 66 done
2024-02-12T16:00:21.666+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-02-12T16:00:22.338+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 122 done
2024-02-12T16:00:22.385+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-02-12T16:00:25.872+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-02-12T16:00:25.990+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-02-12T16:00:27.628+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 63 done
2024-02-12T16:00:28.677+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 97 done
2024-02-12T16:00:29.411+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 121 done
2024-02-12T16:00:29.535+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 94 done
2024-02-12T16:00:30.079+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2948363 jobs created, 2948354 done, 9 pending after 5m0.025097821s (9827/s)
2024-02-12T16:00:30.229+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 104 done
2024-02-12T16:00:38.487+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 120 done
2024-02-12T16:00:39.337+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 119 done
2024-02-12T16:00:40.079+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2948363 jobs created, 2948357 done, 6 pending after 5m10.025307141s (9510/s)
2024-02-12T16:00:40.346+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-02-12T16:00:40.425+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-02-12T16:00:50.079+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2948363 jobs created, 2948359 done, 4 pending after 5m20.025467056s (9213/s)
2024-02-12T16:00:52.314+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 116 done
2024-02-12T16:00:55.373+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-02-12T16:01:00.080+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2948363 jobs created, 2948361 done, 2 pending after 5m30.025637493s (8934/s)
2024-02-12T16:01:10.080+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2948363 jobs created, 2948361 done, 2 pending after 5m40.026334346s (8671/s)
2024-02-12T16:01:16.790+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-02-12T16:01:20.081+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2948363 jobs created, 2948362 done, 1 pending after 5m50.026794091s (8423/s)
2024-02-12T16:01:27.971+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-02-12T16:01:27.972+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 5m57.917540329s
2024-02-12T16:01:27.972+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2948362 results to state tree after 5m57.917581817s
2024-02-12T16:01:27.972+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-02-12T16:01:27.973+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-02-12T16:08:11.818+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All 2948363 done after 12m41.763857225s (3870/s)
migration height  3599392
old cid  bafy2bzacedx4ywdol2hgejjzvgibk43szbelqu67lkfyts43ou6dx7b3md776
new cid  bafy2bzacebfivxial46v2u473bgtdn2ztr2s5oiu2ydzlpcriexblorikabqe
completed round actual (without cache), took  12m44.302997288s
manifest cid: bafy2bzacea6f5icdp6t6fs5sexjxmo3q5d2qu4g4ghq6s5eaob6svnmhvltmw
2024-02-12T16:08:15.006+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-02-12T16:08:15.006+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Started 64 workers
2024-02-12T16:08:15.006+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-02-12T16:12:02.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2948087 migration jobs after 3m47.25647029s
2024-02-12T16:12:02.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 40 done
2024-02-12T16:12:02.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-02-12T16:12:02.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 47 done
2024-02-12T16:12:02.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-02-12T16:12:02.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 42 done
2024-02-12T16:12:02.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 54 done
2024-02-12T16:12:02.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-02-12T16:12:02.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-02-12T16:12:02.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-02-12T16:12:02.262+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-02-12T16:12:02.460+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-02-12T16:12:02.466+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 38 done
2024-02-12T16:12:02.506+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-02-12T16:12:02.530+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-02-12T16:12:02.689+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 43 done
2024-02-12T16:12:02.721+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 50 done
2024-02-12T16:12:02.901+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-02-12T16:12:03.040+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-02-12T16:12:03.288+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 62 done
2024-02-12T16:12:03.294+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-02-12T16:12:03.381+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-02-12T16:12:03.431+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 55 done
2024-02-12T16:12:03.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-02-12T16:12:03.796+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 53 done
2024-02-12T16:12:03.913+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 41 done
2024-02-12T16:12:04.500+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-02-12T16:12:04.880+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-02-12T16:12:05.275+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-02-12T16:12:05.284+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-02-12T16:12:05.447+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 59 done
2024-02-12T16:12:05.503+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-02-12T16:12:05.903+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 51 done
2024-02-12T16:12:06.197+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 45 done
2024-02-12T16:12:06.355+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-02-12T16:12:06.735+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 49 done
2024-02-12T16:12:06.737+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-02-12T16:12:07.239+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-02-12T16:12:07.435+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 48 done
2024-02-12T16:12:07.820+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 44 done
2024-02-12T16:12:07.933+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 56 done
2024-02-12T16:12:08.192+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 52 done
2024-02-12T16:12:08.734+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-02-12T16:12:08.983+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-02-12T16:12:09.340+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-02-12T16:12:09.481+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-02-12T16:12:09.584+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 37 done
2024-02-12T16:12:10.348+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-02-12T16:12:13.244+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 39 done
2024-02-12T16:12:13.492+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-02-12T16:12:16.025+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-02-12T16:12:16.030+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-02-12T16:12:16.030+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-02-12T16:12:16.869+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-02-12T16:12:18.902+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-02-12T16:12:19.546+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-02-12T16:12:22.231+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 58 done
2024-02-12T16:12:28.110+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-02-12T16:12:28.935+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 63 done
2024-02-12T16:12:29.930+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 57 done
2024-02-12T16:12:29.933+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 61 done
2024-02-12T16:12:30.321+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 46 done
2024-02-12T16:12:36.563+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-02-12T16:12:38.985+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-02-12T16:12:42.725+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 60 done
2024-02-12T16:12:42.725+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 4m27.719834011s
2024-02-12T16:12:42.725+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2948086 results to state tree after 4m27.719857375s
2024-02-12T16:12:42.725+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-02-12T16:12:42.726+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-02-12T16:19:24.489+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All 2948087 done after 11m9.482514861s (4404/s)
completed premigration, took  11m12.971836301s
manifest cid: bafy2bzacea6f5icdp6t6fs5sexjxmo3q5d2qu4g4ghq6s5eaob6svnmhvltmw
2024-02-12T16:19:27.339+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Creating migration jobs
2024-02-12T16:19:27.343+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Started 125 workers
2024-02-12T16:19:27.344+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer started
2024-02-12T16:19:37.344+0100	INFO	fil-consensus	filcns/upgrades.go:2418	1099086 jobs created, 1099084 done, 2 pending after 10.004904506s (109855/s)
2024-02-12T16:19:47.344+0100	INFO	fil-consensus	filcns/upgrades.go:2418	2902245 jobs created, 2902245 done, 0 pending after 20.005001588s (145076/s)
2024-02-12T16:19:47.589+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Done creating 2948363 migration jobs after 20.250491184s
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 82 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 116 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 119 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 54 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 114 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 6 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 80 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 102 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 87 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 67 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 111 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 60 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 3 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 83 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 71 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 88 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 55 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 118 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 99 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 78 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 4 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 34 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 53 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 124 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 84 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 27 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 56 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 96 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 85 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 63 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 16 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 75 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 24 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 40 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 20 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 58 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 81 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 62 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 97 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 9 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 77 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 39 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 1 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 33 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 2 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 36 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 18 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 104 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 109 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 66 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 17 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 72 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 115 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 61 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 106 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 41 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 10 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 122 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 117 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 90 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 43 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 31 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 92 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 7 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 45 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 26 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 0 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 59 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 79 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 70 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 12 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 15 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 100 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 25 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 37 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 65 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 76 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 46 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 108 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 73 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 42 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 21 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 74 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 48 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 8 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 35 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 94 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 52 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 47 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 101 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 14 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 51 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 113 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 103 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 11 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 121 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 23 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 29 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 30 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 123 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 93 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 13 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 32 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 91 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 68 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 86 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 57 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 110 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 28 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 89 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 38 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 98 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 105 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 5 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 44 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 49 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 50 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 112 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 69 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 120 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 19 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 22 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 95 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 64 done
2024-02-12T16:19:47.590+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Worker 107 done
2024-02-12T16:19:47.591+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Result writer wrote 2948362 results to state tree after 20.251665956s
2024-02-12T16:19:47.591+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All workers done after 20.251664393s
2024-02-12T16:19:47.591+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migrations
2024-02-12T16:19:47.591+0100	INFO	fil-consensus	filcns/upgrades.go:2418	Running deferred migration for f05
2024-02-12T16:19:55.296+0100	INFO	fil-consensus	filcns/upgrades.go:2418	All 2948363 done after 27.957053542s (105460/s)
completed round actual (with cache), took  30.903692175s


@jennijuju jennijuju added this to the Network v22 milestone Feb 21, 2024
@jennijuju jennijuju added the P1 P1: Must be resolved label Feb 21, 2024
@rjan90 rjan90 changed the base branch from feat/nv22 to release/v1.26.0 March 4, 2024 15:15
@magik6k magik6k marked this pull request as ready for review March 18, 2024 18:48
@magik6k magik6k requested a review from a team as a code owner March 18, 2024 18:48
@magik6k magik6k force-pushed the feat/nv22-migration-bench branch from b14a801 to 135538a Compare March 20, 2024 10:28
@rjan90
Copy link
Contributor

rjan90 commented Mar 21, 2024

Running with the latest go-state-types rc (GST-rc4). The migration was successfull in both the one w/cache, and the one without/cache:

./lotus-shed migrate-state --repo=/home/sealworker1/.lotus 22 bafy2bzaceazc6jg6x4havnmupf5lc33xygdo4zncklpxnd2undcrnjjk4ip42
REMINDER: If you are running this, you likely want to ALSO run the continuity testing tool!
manifest cid: bafy2bzacecoplaet2m4kzueqgutjxpl76bhmuiq5hmo3ueighbnxas3rj4dvy
2024-03-21T07:55:33.203+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Creating migration jobs
2024-03-21T07:55:33.203+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Started 13 workers
2024-03-21T07:55:33.203+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Result writer started
2024-03-21T07:55:43.203+0100	INFO	fil-consensus	filcns/upgrades.go:2420	104849 jobs created, 103836 done, 1013 pending after 10.00032041s (10383/s)
2024-03-21T07:55:53.203+0100	INFO	fil-consensus	filcns/upgrades.go:2420	176511 jobs created, 175498 done, 1013 pending after 20.000447071s (8775/s)
2024-03-21T07:56:03.203+0100	INFO	fil-consensus	filcns/upgrades.go:2420	249005 jobs created, 247992 done, 1013 pending after 30.000661676s (8266/s)
2024-03-21T07:56:13.204+0100	INFO	fil-consensus	filcns/upgrades.go:2420	344437 jobs created, 343424 done, 1013 pending after 40.001499736s (8585/s)
2024-03-21T07:56:23.204+0100	INFO	fil-consensus	filcns/upgrades.go:2420	418749 jobs created, 417736 done, 1013 pending after 50.001854339s (8354/s)
2024-03-21T07:56:33.207+0100	INFO	fil-consensus	filcns/upgrades.go:2420	476630 jobs created, 475617 done, 1013 pending after 1m0.004589973s (7926/s)
2024-03-21T07:56:43.208+0100	INFO	fil-consensus	filcns/upgrades.go:2420	554206 jobs created, 553193 done, 1013 pending after 1m10.004944866s (7902/s)
2024-03-21T07:56:53.208+0100	INFO	fil-consensus	filcns/upgrades.go:2420	612035 jobs created, 611022 done, 1013 pending after 1m20.005125431s (7637/s)
2024-03-21T07:57:03.208+0100	INFO	fil-consensus	filcns/upgrades.go:2420	679851 jobs created, 678838 done, 1013 pending after 1m30.005387585s (7542/s)
2024-03-21T07:57:13.209+0100	INFO	fil-consensus	filcns/upgrades.go:2420	758859 jobs created, 757846 done, 1013 pending after 1m40.005993754s (7578/s)
2024-03-21T07:57:23.210+0100	INFO	fil-consensus	filcns/upgrades.go:2420	816328 jobs created, 815315 done, 1013 pending after 1m50.007443915s (7411/s)
2024-03-21T07:57:33.210+0100	INFO	fil-consensus	filcns/upgrades.go:2420	899880 jobs created, 898867 done, 1013 pending after 2m0.007563508s (7490/s)
2024-03-21T07:57:43.213+0100	INFO	fil-consensus	filcns/upgrades.go:2420	954299 jobs created, 953286 done, 1013 pending after 2m10.010770629s (7332/s)
2024-03-21T07:57:53.214+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1037712 jobs created, 1036699 done, 1013 pending after 2m20.011518866s (7404/s)
2024-03-21T07:58:03.214+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1095842 jobs created, 1094829 done, 1013 pending after 2m30.011906692s (7298/s)
2024-03-21T07:58:13.215+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1158892 jobs created, 1157879 done, 1013 pending after 2m40.012383758s (7236/s)
2024-03-21T07:58:23.216+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1206465 jobs created, 1205452 done, 1013 pending after 2m50.013218019s (7090/s)
2024-03-21T07:58:33.217+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1253551 jobs created, 1252538 done, 1013 pending after 3m0.013970367s (6958/s)
2024-03-21T07:58:43.217+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1325873 jobs created, 1324864 done, 1009 pending after 3m10.014401223s (6972/s)
2024-03-21T07:58:53.217+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1408237 jobs created, 1407224 done, 1013 pending after 3m20.014767838s (7036/s)
2024-03-21T07:59:03.217+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1484741 jobs created, 1483728 done, 1013 pending after 3m30.01488481s (7065/s)
2024-03-21T07:59:13.218+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1554679 jobs created, 1553666 done, 1013 pending after 3m40.015113742s (7062/s)
2024-03-21T07:59:23.218+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1633133 jobs created, 1632120 done, 1013 pending after 3m50.015243634s (7096/s)
2024-03-21T07:59:33.218+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1701743 jobs created, 1700730 done, 1013 pending after 4m0.015371951s (7086/s)
2024-03-21T07:59:43.220+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1778327 jobs created, 1777314 done, 1013 pending after 4m10.017613958s (7109/s)
2024-03-21T07:59:53.220+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1857408 jobs created, 1856395 done, 1013 pending after 4m20.017860453s (7139/s)
2024-03-21T08:00:03.224+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1923498 jobs created, 1922485 done, 1013 pending after 4m30.02140947s (7120/s)
2024-03-21T08:00:13.225+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1983252 jobs created, 1982239 done, 1013 pending after 4m40.021969551s (7079/s)
2024-03-21T08:00:23.226+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2035826 jobs created, 2034813 done, 1013 pending after 4m50.02337426s (7016/s)
2024-03-21T08:00:33.229+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2099477 jobs created, 2098464 done, 1013 pending after 5m0.026640339s (6994/s)
2024-03-21T08:00:43.230+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2155272 jobs created, 2154414 done, 858 pending after 5m10.027352617s (6949/s)
2024-03-21T08:00:53.230+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2223198 jobs created, 2222185 done, 1013 pending after 5m20.027507213s (6944/s)
2024-03-21T08:01:03.230+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2281483 jobs created, 2280470 done, 1013 pending after 5m30.027737449s (6910/s)
2024-03-21T08:01:13.230+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2330971 jobs created, 2329958 done, 1013 pending after 5m40.02784708s (6852/s)
2024-03-21T08:01:23.231+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2384824 jobs created, 2383811 done, 1013 pending after 5m50.027999927s (6810/s)
2024-03-21T08:01:33.231+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2434183 jobs created, 2433171 done, 1012 pending after 6m0.02816306s (6758/s)
2024-03-21T08:01:43.231+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2485071 jobs created, 2484058 done, 1013 pending after 6m10.028558484s (6713/s)
2024-03-21T08:01:53.232+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2555988 jobs created, 2554975 done, 1013 pending after 6m20.029540622s (6723/s)
2024-03-21T08:02:03.232+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2632091 jobs created, 2631078 done, 1013 pending after 6m30.02991246s (6746/s)
2024-03-21T08:02:13.234+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2700974 jobs created, 2699961 done, 1013 pending after 6m40.031717153s (6749/s)
2024-03-21T08:02:23.235+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2777804 jobs created, 2776798 done, 1006 pending after 6m50.032148326s (6772/s)
2024-03-21T08:02:33.235+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2828150 jobs created, 2827137 done, 1013 pending after 7m0.032286468s (6731/s)
2024-03-21T08:02:43.235+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2889863 jobs created, 2888850 done, 1013 pending after 7m10.032426226s (6718/s)
2024-03-21T08:02:53.238+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2940885 jobs created, 2939872 done, 1013 pending after 7m20.03567846s (6681/s)
2024-03-21T08:03:03.239+0100	INFO	fil-consensus	filcns/upgrades.go:2420	3022815 jobs created, 3021802 done, 1013 pending after 7m30.036201799s (6715/s)
2024-03-21T08:03:03.417+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Done creating 3025133 migration jobs after 7m30.214019845s
2024-03-21T08:03:03.453+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 11 done
2024-03-21T08:03:03.483+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 4 done
2024-03-21T08:03:03.603+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 3 done
2024-03-21T08:03:03.663+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 9 done
2024-03-21T08:03:03.691+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 1 done
2024-03-21T08:03:03.772+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 7 done
2024-03-21T08:03:04.087+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 2 done
2024-03-21T08:03:04.913+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 12 done
2024-03-21T08:03:05.061+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 8 done
2024-03-21T08:03:05.261+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 0 done
2024-03-21T08:03:05.337+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 6 done
2024-03-21T08:03:05.765+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 10 done
2024-03-21T08:03:06.435+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 5 done
2024-03-21T08:03:06.436+0100	INFO	fil-consensus	filcns/upgrades.go:2420	All workers done after 7m33.232954785s
2024-03-21T08:03:06.436+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Result writer wrote 3025132 results to state tree after 7m33.232968841s
2024-03-21T08:03:06.436+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Running deferred migrations
2024-03-21T08:03:06.436+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Running deferred migration for f05
2024-03-21T08:10:33.655+0100	INFO	fil-consensus	filcns/upgrades.go:2420	All 3025133 done after 15m0.45263022s (3360/s)
migration height  3754689
old cid  bafy2bzacebvrukaawjrlievkx62v3gu32n6ic47irud442ujdulrh7pf5jyay
new cid  bafy2bzaceato35nutx377l5hl3jgfguw4zwbgilivljwdn267ohyo7ta6hzuo
completed round actual (without cache), took  15m3.251168576s
manifest cid: bafy2bzacecoplaet2m4kzueqgutjxpl76bhmuiq5hmo3ueighbnxas3rj4dvy
2024-03-21T08:10:36.577+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Creating migration jobs
2024-03-21T08:10:36.577+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Started 8 workers
2024-03-21T08:10:36.577+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Result writer started
2024-03-21T08:15:36.577+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1706454 jobs created, 1706446 done, 8 pending after 5m0.000149039s (5688/s)
2024-03-21T08:19:45.753+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Done creating 3024563 migration jobs after 9m9.176171912s
2024-03-21T08:19:45.753+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 5 done
2024-03-21T08:19:45.753+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 4 done
2024-03-21T08:19:45.763+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 2 done
2024-03-21T08:19:45.904+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 3 done
2024-03-21T08:19:45.986+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 7 done
2024-03-21T08:19:46.015+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 0 done
2024-03-21T08:19:46.759+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 6 done
2024-03-21T08:19:47.013+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 1 done
2024-03-21T08:19:47.013+0100	INFO	fil-consensus	filcns/upgrades.go:2420	All workers done after 9m10.435621883s
2024-03-21T08:19:47.013+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Result writer wrote 3024562 results to state tree after 9m10.435653232s
2024-03-21T08:19:47.013+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Running deferred migrations
2024-03-21T08:19:47.013+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Running deferred migration for f05
2024-03-21T08:27:17.269+0100	INFO	fil-consensus	filcns/upgrades.go:2420	All 3024563 done after 16m40.691516511s (3022/s)
completed premigration, took  16m43.748597977s
manifest cid: bafy2bzacecoplaet2m4kzueqgutjxpl76bhmuiq5hmo3ueighbnxas3rj4dvy
2024-03-21T08:27:20.212+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Started 13 workers
2024-03-21T08:27:20.212+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Result writer started
2024-03-21T08:27:20.212+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Creating migration jobs
2024-03-21T08:27:30.212+0100	INFO	fil-consensus	filcns/upgrades.go:2420	1260114 jobs created, 1260114 done, 0 pending after 10.000181582s (126009/s)
2024-03-21T08:27:40.213+0100	INFO	fil-consensus	filcns/upgrades.go:2420	2801535 jobs created, 2801535 done, 0 pending after 20.000562002s (140073/s)
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Done creating 3025133 migration jobs after 21.63416519s
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 8 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 6 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 2 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 5 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 10 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 7 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 4 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 3 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 9 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 1 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 0 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 12 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Worker 11 done
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Result writer wrote 3025132 results to state tree after 21.634383173s
2024-03-21T08:27:41.846+0100	INFO	fil-consensus	filcns/upgrades.go:2420	All workers done after 21.634377783s
2024-03-21T08:27:41.847+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Running deferred migrations
2024-03-21T08:27:41.847+0100	INFO	fil-consensus	filcns/upgrades.go:2420	Running deferred migration for f05
2024-03-21T08:27:45.523+0100	INFO	fil-consensus	filcns/upgrades.go:2420	All 3025133 done after 25.310458968s (119521/s)
completed round actual (with cache), took  29.655731352s

@rjan90 rjan90 merged commit 734db29 into release/v1.26.0 Mar 21, 2024
93 of 94 checks passed
@rjan90 rjan90 deleted the feat/nv22-migration-bench branch March 21, 2024 14:07
rjan90 pushed a commit that referenced this pull request Mar 22, 2024
* shed: Add v13 migration to migrate-state

* shed: some ADL tools, update GST

* shed: diff hamt address mode

* shed migration debug tooling

* shed migration: market diff on error

* shed: Fix cached migration diff

* shed: Diff deal states on bad migration

* shed: Use std json

* shed: Drill in the migration diff more

* shed: Show proposals in migration market diff

* shed: Show added provider sectors diff

* shed: hamts are hard to use

* update  gst

* update gst

* update gst

* update GST with fixed invartiant

* go mod tidy
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 P1: Must be resolved
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

7 participants